System to process load cell data using door sensor data

ABSTRACT

Shelves or other fixtures may be used to hold items at a facility. Load cells at the fixtures may be used to acquire weight data indicative of changes to the fixture as items are added or removed from the fixture. The fixtures, such as a freezer case, may include a door that when opened or closed produces air movement with respect to the shelves. This air movement may produce weight data that could be interpreted as a false event, such as a pick or place, even when none has taken place. A door sensor provides data that is used to determine when the door has been opened or closed. Weight data occurring for some time after the opening of the door or before the closing of the door may be disregarded.

BACKGROUND

Retailers, wholesalers, and other product distributors typicallymaintain an inventory of various items that may be ordered, purchased,leased, borrowed, rented, viewed, and so forth, by clients or customers.For example, an e-commerce website may maintain inventory in afulfillment center. When a customer orders an item, the item is pickedfrom inventory, routed to a packing station, packed, and shipped to thecustomer. Likewise, physical stores maintain inventory in customeraccessible areas, such as in a shopping area, and customers can pickitems from inventory and take them to a cashier for purchase, rental,and so forth.

Many physical stores also maintain inventory in a storage area,fulfillment center, or other facility that can be used to replenishinventory located in the shopping areas or to satisfy orders for itemsthat are placed through other channels (e.g., e-commerce). Otherexamples of entities that maintain facilities holding inventory includelibraries, museums, rental centers, and so forth. In each instance, foran item to be moved from one location to another, it is picked from itscurrent location and transitioned to a new location. It is oftendesirable to monitor quantity of inventory within the facility.

BRIEF DESCRIPTION OF FIGURES

The detailed description is set forth with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical items or features. The figures are not necessarily drawn toscale, and in some figures, the proportions or other aspects may beexaggerated to facilitate comprehension of particular aspects.

FIG. 1 illustrates a system that processes load cell data to producereliable interaction data about the pick (removal) or place (return orloading) of items stowed on a fixture, according to someimplementations.

FIG. 2 is an overhead view of a fixture, such as a freezer case, thatincludes shelves with load cells and a door, according to someimplementations.

FIG. 3 depicts a graph of weight data and door sensor data that are usedto determine when weight data from load cells is valid, according tosome implementations.

FIG. 4 is a flow diagram illustrating a process to determine interactiondata based on weight data that has been determined to be valid,according to some implementations.

FIG. 5 depicts graphs of weight values from multiple shelves and acorrespondence between those weight values to determine when weight datafrom load cells is valid, according to some implementations.

FIG. 6 is a flow diagram illustrating a process to determine validity ofevents based on weight data using a correspondence between weight datafrom multiple shelves, according to some implementations.

FIG. 7 depicts a graph of a two-dimensional space and a trajectory of anestimated location of a center of mass or other characteristic of ashelf that may be used to determine when weight data from load cells isvalid, according to some implementations.

FIG. 8 depicts a graph of a three-dimensional space and a trajectory ofan estimated location of a center of mass or other characteristic of ashelf that may be used to determine when weight data from load cells isvalid, according to some implementations.

FIG. 9 is a flow diagram illustrating a process that uses an estimatedlocation of a center of mass or other characteristic of a shelf todetermine if weight data is valid, according to some implementations.

FIG. 10 is a flow diagram illustrating a process that uses a trajectorybased on weight data from load cells at a shelf to determine if weightdata is valid, according to some implementations.

FIG. 11 is a block diagram of a module to determine validity of weightdata from load cells, according to some implementations.

FIG. 12 is a block diagram illustrating a materials handling facility(facility) using the system, according to some implementations.

FIG. 13 is a block diagram illustrating additional details of thefacility, according to some implementations.

FIG. 14 is a block diagram of a server to support operation of thefacility, according to some implementations.

While implementations are described herein by way of example, thoseskilled in the art will recognize that the implementations are notlimited to the examples or figures described. It should be understoodthat the figures and detailed description thereto are not intended tolimit implementations to the particular form disclosed but, on thecontrary, the intention is to cover all modifications, equivalents, andalternatives falling within the spirit and scope as defined by theappended claims. The headings used herein are for organizationalpurposes only and are not meant to be used to limit the scope of thedescription or the claims. As used throughout this application, the word“may” is used in a permissive sense (i.e., meaning having the potentialto), rather than the mandatory sense (i.e., meaning must). Similarly,the words “include,” “including,” and “includes” mean including, but notlimited to.

DETAILED DESCRIPTION

A materials handling facility (facility) or other setting may utilizefixtures, such as shelves, that are equipped with load cells or otherweight sensing devices. These fixtures may include an enclosure with adoor, within which the shelves are located. For example, a freezer casemay include an insulated enclosure with a door. In some implementations,the door may be separate from the enclosure. A user of the facility mayopen the door to access the shelves and items stowed on those shelves.

The shelves may be mounted to, or otherwise include, load cells thatmeasure the weight of a load on the shelf. For example, the fixture maycomprise rectangular shelves, with each shelf including load cellsarranged at or near each end of the shelf, at or near each of the fourcorners, or in other configurations.

Weight data generated by these load cells may be used to determineinteractions with items stowed or otherwise held by those fixtures. Thefacility may include, or have access to, an inventory management system.The inventory management system may be configured to maintaininformation about items, users, condition of the facility, and so forth.For example, the inventory management system may maintain dataindicative of a number of items at a particular fixture, what items aparticular user is ordered to pick, how many items have been picked orplaced at the fixture, requests for assistance, environmental status ofthe facility, and so forth.

Operation of the facility may be facilitated by using data from sensors,such as the load cells, to determine interactions in the facility.Interactions may comprise the user picking an item from a fixture,placing an item at a fixture, touching an item, bringing an object suchas a hand or face close to an item, and so forth. For example, theinventory management system may generate interaction data that indicateswhat item a user picked from a particular fixture, and then use thisinteraction data to adjust the count of inventory stowed at theparticular fixture.

The weight data may be used to determine data such as a quantity ofitems that have been picked from or placed to a fixture, to determinethe identity of a type of item that has been picked from or placed tothe fixture, and so forth. Use of load cells and the weight data offersseveral operational benefits, especially in a materials handlingfacility or other facility. These benefits may include mechanicalrobustness, relatively low cost for installation and maintenance, fastresponse times, and so forth.

As described above, the fixture may include a shelf or other structurethat is supported by load cells that generate the weight data. The loadcells may be sensitive enough such that air movement may cause atransitory change in the weight data being output. For example, when thedoor of the fixture is opened or closed, some volume of air isdisplaced. The displacement of this volume of air may result inerroneous weight data being generated by the load cells that indicates achange in weight when in fact, no change has taken place. When thefixture involves a refrigerated or heated interior, thermal effects maycause additional air movement, exacerbating this issue. For example,when the door of a freezer case is opened, movement of the door from aclosed configuration to an open configuration displaces a mass of airwhich impacts the shelves and results in changes to the weight data.Continuing the example, once the door opens, cold air may spill from theinterior of the fixture drawing warm air into the enclosure. The mass ofair thus moved may result in some displacement of the shelf.

Erroneous weight data, if not accounted for, could result in erroneousinteraction data, increased use of processing resources, and so forth.For example, the transitory change in the values of the weight data thatresult from the displacement of air after opening a door could result inthe inventory management system generating interaction data indicativeof a pick or place of an item when no such pick took place. In anotherexample, the transitory change may trigger the system to presentinformation to a human operator to determine if an interaction actuallytook place. As a result, processing of erroneous weight data may resultin errors which would then have to be addressed, unnecessary consumptionof resources, and so forth.

Described in this disclosure are techniques to determine whether theweight data produced by one or more load cells is deemed to be valid ornot. Valid data comprises weight data that is not deemed to be likely tobe produced by an air movement or mechanism other than the user. Incomparison, invalid data is indicative of weight data that is deemedlikely to be erroneous and thus should be disregarded from determinationof interaction data.

In a first technique, the fixture may incorporate a door sensor. Thedoor sensor provides door sensor data that is indicative of aconfiguration of the door at a particular time. For example, the doorsensor may comprise a switch that breaks an electric circuit when thedoor is closed and makes an electric circuit when the door is open.Electronics may use the operation of the switch to generate the doorsensor data. For example, the door sensor data may comprise a single bitbinary value in which a “0” indicates that the door is closed and a “1”indicates that the door is open.

The door sensor data may be used to determine validity data indicativeof whether the weight data is valid or invalid at a specific time orinterval of time. The weight data may be deemed invalid if it wasobtained during a time window that starts at a time when the door wasopened and continues for a predetermined period of time thereafter. Forexample, at t=0, the door was opened. A predetermined time interval of 1second may be specified. Continuing the example, the weight dataobtained from t=0 until t=1 second may be deemed invalid. In anotherexample, the weight data obtained whenever the door sensor dataindicates the doors closed may be deemed valid. With thisimplementation, the actual values of the weight data may not beconsidered to determine validity. For example, the validity data isdetermined based on the time the values of the weight data were obtainedand the information about the state of the door. In another example, thevalidity data may comprise information about the state of the door.

In a second technique, validity data may be determined using the weightdata itself. During normal operation of a facility, a user may beexpected to pick or place items from the shelves of the fixture. It maybe deemed unlikely for simultaneous picks or places to take place atmultiple shelves. In comparison, influences such as changes due to airmovement when a door is opened or closed may affect several shelves atabout the same time. In this implementation, the weight data is analyzedto determine if a correspondence between the weight data of the adjacentshelves at a particular window of time exceeds a threshold value. Forexample, the weight data from a top shelf and the weight data from thebottom shelf may be correlated to determine a correlation value. If thecorrelation value exceeds a threshold amount, validity data may begenerated that indicates invalid weight data. If the correlation valueis less than the threshold amount, validity data may be generated thatindicates the weight data is valid.

Additional techniques may be used to determine validity data. Thesetechniques may be used to determine if the weight data, or datagenerated from the weight data, is outside of an expected value or rangeof value. The values may be compared in a space which may correspond toa physical space, such as within the confines of the fixture or aportion thereof, or within in an arbitrary space having one or moredimensions.

A third technique that may be used to analyze the weight data todetermine validity data that looks for conditions in the weight datathat are deemed to be physically impossible, or at least physicallyextremely unlikely. The weight data may be accessed and used todetermine an estimated location of a center of mass, estimated locationof a weight change, or other data that is indicative of a physicallocation. The estimated location may be compared to a physical boundaryof the fixture, such as a perimeter of the shelf. Influences on theshelf, such as the changes due to air movement when a door is opened orclosed may result in a change in the estimated location at a given time.For example, a mass of air that impacts a leading edge of the shelf maytemporarily increase the values of the weights measured by the loadcells at the front of the shelf. As a result, at the time when the airimpacts the front of the shelf, the estimated location of the center ofmass of the shelf or an estimated location of a weight change may beerroneously determined to be several centimeters beyond the front edgeof the shelf. If the estimated location is within the physical boundaryof the shelf, validity data may be generated that indicates valid weightdata. In comparison, if the estimated location is outside the physicalboundary, validity data may be generated that indicates invalid weightdata.

In a fourth technique, trajectory data may be determined from the weightdata. For example, the trajectory data may comprise a time series of theweight data, a time series of values derived from the weight data, andso forth. The trajectory data may be expressed as points within a spacethat has one or more dimensions. The trajectory data describes atrajectory, or path through the space. The space may be arbitrary, suchthat a point within the space does not necessarily correspond with apoint in real three-dimensional space. The trajectory describes a paththrough this space that may be arbitrary and may not correspond tomotion in real three-dimensional space. Within the space, one or moreregions may be designated as being consistent with invalid weight data.The region may be an interval in the case of a one-dimensional space, anarea in the case of a two-dimensional space, a volume in the case of athree-dimensional space, and so forth. If the trajectory data indicatesthat the trajectory produced from the weight data does not intersect theparticular region, validity data may be generated that indicates validweight data. Likewise, if the trajectory data indicates that thetrajectory produced from the weight data intersects the particularregion, validity data may be generated that indicates invalid weightdata. In one implementation, boundaries of the region(s) may bespecified. In another implementation, a machine learning module may betrained to characterize the trajectory data and provide the validitydata.

One or more of the techniques may be combined to provide a robustdetermination as to the validity of the weight data. For example, thefixture that includes a door sensor may produce door sensor data andweight data that are used as inputs to train a machine learning module.In the event of a failure of the door sensor, or of a fixture of similarconfiguration but which omits the door sensor, the machine learningmodule may be used to generate door status data using only weight dataas input. In another example, the determination of correspondencesbetween different shelves may be combined with the analysis oftrajectory data to provide more robust door status data.

By using the devices and techniques described herein, operation of thefacility may be improved. Invalid weight data containing noise isquickly and easily recognized as such and disregarded from considerationwhile valid data may be used to determine accurate interaction data. Thereduction in invalid or false data reduces overall resource utilizationby avoiding further processing either by computing devices or by havinga human review. In comparison, valid weight data may be used todetermine accurate interaction data.

By determining a fault in a load cell, the inventory management systemis able to disregard erroneous data and avoid generating output based onfaulty data. As a result, the inventory management system may be able toquickly and efficiently track what item a user has interacted with,maintain up-to-date item data, and so forth.

Illustrative System

FIG. 1 illustrates a system 100 that processes load cell data to producereliable interaction data about a fixture, such as a shelf, according tosome implementations.

A fixture 102 may include one or more shelves 104. In this illustration,four shelves 104(1)-104(4) are depicted for illustration only and notnecessarily as a limitation. In other implementations, the fixture 102may enclose greater or fewer numbers of shelves 104. The shelves 104 areconfigured to stow one or more items 106. In some implementations,instead of or in addition to shelves 104, other item stowage devicessuch as hangers, bins, and so forth, may be used.

The shelves 104 may be equipped with a plurality of load cells 108. Eachload cell 108 is configured to generate information indicative of aweight applied thereto. For example, the load cell 108 may comprise aweight sensor that supports at least a portion of the fixture 102. Theload cells 108 are described in more detail below.

The fixture 102 may comprise an enclosure, with the shelves 104 locatedwithin. The fixture 102 may include a door 110 that provides a barrierbetween an interior of the enclosure and the exterior environment. Forexample, the fixture 102 may comprise an insulated enclosure and arefrigeration mechanism to provide a cold case for the storage ofchilled or frozen items 106 therein. The refrigeration mechanism maycomprise Peltier junctions, passive heat transfer system using arefrigerant or thermally conductive element, active heat transfer systemusing a pump that circulates a refrigerant through a heat exchanger, andso forth. In another example, the fixture 102 may comprise a “hot case”or an insulated enclosure and heating devices, such as incandescent heatlamps or heating elements, to provide for the storage of warm or hotitems 106 therein. In some implementations, fans, blowers, or otherdevices may be used to move air within the fixture 102, resulting in airmovement 116. For example, the heating elements may compriseelectrically resistive ceramic elements such as molybdenum disilicide,barium titanate, electrically resistive wire such as nichrome, burners,and so forth.

The door 110 may be moved between a closed configuration 110(1) and anopen configuration 110(2). In the closed configuration 110(1), the door110 provides a barrier between an interior of the fixture 102 and theexterior environment. In the open configuration 110(2), the door 110 nolonger provides this barrier.

During operation of the facility, a user 112 may approach the fixture102 and open the door 110 to access one or more of the items 106 stowedtherein on one or more shelves 104. Once the user 112 has completedtheir interaction with the items 106, the door 110 may close.

In some implementations, the fixture 102 may include a door sensor 114.The door sensor 114 provides information indicative of the configurationof the door 110. The door sensor 114 may utilize any number of differentdevices to determine a state of the door 110.

In some implementations, the door sensor 114 may comprise one or more ofa switch, a capacitive proximity sensor, an ultrasonic proximity sensor,an optical proximity sensor, a magnetic field sensor, an anemometer, atemperature sensor, a potentiometer, a force sensitive resistor, anoptical encoder, a camera, and so forth. For example, the fixture 102may include a switch that is configured to make or break a contact inresponse to a mechanical force provided by the door 110. Continuing theexample, the switch may be spring-loaded such that, when the door 110 isopen, a contact is made and electric circuit completed and when the door110 is closed, the contact is opened or broken. In another example, acamera may have a field of view that includes at least a portion of thedoor. The camera may be mounted to the fixture 102, may be inside thefixture 102, may be overhead, on a wall, on another fixture 102, and soforth. The images obtained from the camera may be processed to determinethe position of the door 110 with respect to the fixture 102. Forexample, an object recognition module may be used to detect the door 110and determine if the door 110 is open or closed. In another example, thedoor 110 may include an optical target, such as a particular visualfiducial, barcode, and so forth. The optical target may be detectedwithin the image data and used to determine the door sensor data 124.

The door sensor 114 may be mounted on the fixture 102, on a shelf 104,on a structure supporting the shelf 104, on the door 110, and so forth.In some implementations, the door sensor 114 may be proximate to or atleast partially within a handle of the door 110. For example, the doorsensor 114 may comprise a pressure sensor or pressure contact switchthat detects a hand of the user 112 grasping the handle. In anotherexample, the door sensor 114 may comprise a proximity sensor thatdetects the presence of an object near the handle of the door 110. Inyet another example the door sensor 114 may comprise a touch sensor,such as a galvanic, capacitive, inductive, or resistive circuit, thatdetects a touch of a user's hand to the handle of the door 110. Theresulting data produced by the door sensor 114 associated with thehandle is thus indicative of use of the handle.

Air movement 116 or other environmental influences such as vibrationfrom movement of the door 110, a nearby train, and so forth may affectoperation of the load cells 108. For example, as the door 110 is openedor closed, movement of the door 110 may result in a displacement of somemass of air. This mass of air may come in contact with at least aportion of a shelf 104 temporarily applying a force that is registeredby the load cell 108 as a change in weight. However, this effect istypically transitory and does not reflect an actual change in the loadof the items 106 upon the shelf 104.

When the fixture 102 involves a refrigerated or heated interior, thermaleffects may cause additional air movement, exacerbating this issue. Forexample, when the door 110 of a freezer case is opened, movement of thedoor 110 from a closed configuration door 110(1) to an openconfiguration door 110(2) displaces a mass of air which impacts theshelves 104 and results in changes to the weight data. Continuing theexample, in addition to the displacement of air caused by the movementof the door 110, once the door 110 is open the denser cold air may spillfrom the interior of the fixture 102, drawing warm air into theenclosure. The mass of air thus moved may result in some displacement ofthe shelf 104. This air movement 116 is depicted in this figure usingarrows drawn with dotted lines. In other implementations, devices suchas blowers or fans may produce air movement 116 within at least aportion of the interior of the fixture 102.

The fixture 102 may include one or more electronic devices that aredesigned to acquire fixture data 118, and provide that fixture data 118to an inventory management system 120. The fixture data 118 may includeone or more of weight data 122, door sensor data 124, or other data fromsensors in or on the fixture 102.

The weight data 122 is indicative of the weight as measured by the loadcells 108. For example, the load cells 108 may send weight signals to acomputing device of the fixture 102 that then generates the weight data122, which includes weight values and timestamps indicative of whenthose weight values were obtained. In some implementations, thecomputing device of the fixture 102 may provide various functions, suchas filtering or otherwise conditioning the weight signals, generatingand transmitting the weight data 122, and so forth.

The door sensor data 124 is indicative of output from the door sensor114. For example, the fixture 102 may include electronics that generatedoor sensor data 124 by determining if a switch acting as the doorsensor 114 is open or closed.

In some implementations, the door sensor data 124 may be inserted into aportion of the weight data 122. For example, the door sensor data 124may comprise a single bit binary value in which a “0” indicates the door110 is closed and a “1” indicates the door 110 is open. Multiplexingtechniques may be used to replace a least significant bit from theweight data 122 with the single bit binary value of the door sensor data124, generating multiplexed data. In one implementation, a computingdevice, hardware multiplexor, or other device may be configured toaccess the weight data 122 and replace one or more bits of the weightdata 122 with the sensor data from the door sensor 114 to createmultiplexed data. This multiplexed data may then be transmitted using acommunication interface. For example, a computing device at the fixture102 that acquires the weight data 122 from the load cells 108 and thedoor sensor data 124 from the door sensor 114 and transmits that fixturedata 118 using a network may be configured to provide this operation. Byencoding the door sensor data 124 as a least significant bit anysubsequent processing of the weight data 122 that is not configured toutilize or disregard that bit does not produce significant changes inthe operation of the system. Additionally, by encoding the door sensordata 124 in this fashion, subsequent processing is able to readilydetermine the door state with a granularity that may extend toindividual samples of weight data 122.

The inventory management system 120 may be configured, as describedbelow, to perform various functions with regard to a facility. Toperform these functions, the inventory management system 120 may acceptthe fixture data 118, data from other sensors, and so forth.

The inventory management system 120 may include or have access to ananalysis module 126. The analysis module 126 is configured to accept asinput fixture data 118 and determine validity data 128. The validitydata 128 provides information indicative of whether at least a portionof the weight data 122 is deemed to be valid or invalid. For example,the validity data 128 may indicate that a first portion of the weightdata 122 that was obtained between a first time and a second time isvalid, while a second portion of the weight data 122 that was obtainedbetween a third time and a fourth time is invalid.

The analysis module 126 may utilize one or more techniques to determinethe validity data 128. A first technique utilizes the door sensor data124. For example, the weight data 122 associated with a particularinterval of time that includes the door 110 opening or closing may bedeemed to be invalid. A second technique determines data about thecorrespondence between the weight data 122 from a plurality of shelves104 during a particular interval of time to determine the validity data128. For example, within the particular interval of time, the airmovement 116 may similarly affect the weight data 122 generated by theshelves 104(1)-(4), resulting in a high correlation between the weightdata 122 from the respective shelves 104. This high correlation may beindicative of invalid weight data 122 during that particular interval oftime. A third technique uses an estimated location of the center of massof a shelf 104 to determine the validity data 128. For example, if theestimated location of a center of mass of the shelf 104 is beyond thephysical boundary of the shelf 104, the weight data 122 during thatparticular interval of time may be deemed to be invalid. A fourthtechnique determines trajectory data from the weight data 122 todetermine the validity data 128. These techniques are described in moredetail at least in FIGS. 3-11.

The inventory management system 120 may maintain and utilize item data130 and physical layout data 132. The item data 130 comprisesinformation about a particular type of item 106. The item data 130 mayinclude information indicative of a weight of a single item 106, or apackage, kit, or other grouping considered to be a single item 106. Theitem data 130 may include other characteristics of that type of item 106such as: physical dimensions, characteristics about how the item 106appears, and so forth. For example, the item data 130 may comprise aplurality of local descriptor values generated by feature extractionalgorithms, parameters for classifiers, neural network configurationdata, and so forth, that characterizes the appearance of arepresentative one or more of the item 106. The item data 130 mayindicate the types and quantities of items 106 that are expected to bestored at that particular fixture 102, such as in a particular lane on afixture 102. The item data 130 may include other data. For example, theother data may comprise weight distribution of the item 106, point clouddata for the item 106, and so forth.

The physical layout data 132 may provide information indicative of wherefixtures 102 are in the facility, location of sensors, information aboutsensor orientation and field of view (where applicable), and so forth.For example, the physical layout data 132 may comprise informationrepresentative of a map or floor plan of the facility with relativepositions of fixtures 102, planogram data indicative of how types ofitems 106 are to be arranged at the fixtures 102, and so forth. Inanother example, the physical layout data 132 may comprise informationindicating the particular placement of load cells 108 on a particularfixture 102.

During operation, the analysis module 126 may access one or more of thefixture data 118, weight data 122, the door sensor data 124, the itemdata 130, the physical layout data 132, or other data to generateinteraction data 134. For example, the facility may include othersensors 136, such as cameras having a field of view that includes atleast a portion of the fixture 102. The sensors 136 generate sensordata, such as image data from the camera.

The interaction data 134 provides information about an interaction, suchas a pick of an item 106 from the fixture 102, a place of an item 106 tothe fixture 102, a touch made to an item 106 at the fixture 102, agesture associated with an item 106 at the fixture 102, and so forth.The interaction data 134 may include one or more of the type ofinteraction, interaction location identifier indicative of the fixture102 at which the interaction took place, an item identifier indicativeof a type of item 106 or particular item 106, quantity change to theitem 106, user identifier, and so forth. The interaction data 134 maythen be used to further update the item data 130. For example, thequantity of items 106 on hand at a particular partitioned area on thefixture 102 may be changed based on an interaction that picks or placesone or more items 106. Operation of the analysis module 126, includinggeneration of the interaction data 134, is discussed in more detailbelow.

In order to provide interaction data 134 with the best accuracy, theanalysis module 126 should be provided with weight data 122 that is noterroneous or invalid. As described above, air movements 116 or otherphenomena may result in the weight data 122 including invalid data.

As described above, the analysis module 126 may be configured to assessthe weight data 122 and generate validity data 128. This validity data128 may be used to assess the weight data 122 prior to processing todetermine interaction data 134. For example, weight data 122 that isindicated by the validity data 128 as being invalid may be disregardedand not processed to determine interaction data 134 while weight data122 that is indicated by the validity data 128 as being valid may beprocessed and used to determine the interaction data 134. In this way,the analysis module 126 avoids processing erroneous weight data 122 thatcould lead to incorrect interaction data 134 being generated.

The analysis module 126, or other modules, may process the weight data122 to determine if an event has taken place at the fixture 102. Eventsmay include a pick or place of an item 106. One or more change detectionalgorithms may be used to identify when the weight values have changedin such a way as to meet or exceed a threshold value. For example, theevent may be determined using a cumulative sum (CUSUM) function. Anevent may include weight data 122 that begins at a start time andcontinues through an end time. In some implementations, an event may becategorized as valid or invalid based on the validity data 128associated with the associated weight data 122.

By using these techniques, the amount of erroneous weight data 122 thatis processed is reduced improving overall system efficiency and reducingcomputing resource needs. Additionally, by validating the weight data122, the accuracy of the interaction data 134 is improved, improvingoverall accuracy of the inventory management system 120 and the facilityas a whole.

FIG. 2 is an overhead view 200 of a fixture 102, such as a freezer case,that includes shelves 104 with load cells 108 and a door 110, accordingto some implementations.

As depicted here, the weight of a shelf 104 in the fixture 102 may bemeasured using four load cells 108(1), 108(2), 108(3), and 108(4). Eachload cell 108 may be located at the four corners of the rectangularfixture 102. For example, the load cell 108(1) is located at a rear leftcorner, the load cell 108(2) is located at a front left corner, the loadcell 108(3) is located at a front right corner, and the load cell 108(4)is located at a rear right corner. In other implementations, otherarrangements of load cells 108 may be utilized.

The fixture 102 may be divided into one or more partitioned areas 202.The partitioned areas 202 are an area upon the shelf 104 that isassociated with a particular type of item 106. For example, the shelf104 depicted here has three partitioned areas 202(1), 202(2), and202(3). Partitioned area 202(1) may be associated with storing somequantity of item 106(1) such as frozen fish, while partitioned area202(2) may be associated with storing some quantity of item 106(2) suchas frozen vegetables, and partitioned area 202(3) may be associated withstoring some quantity of item 106(3) such as frozen fruit.

A user 112 may interact with the partitioned areas 202 at the shelf 104.For example, the user 112 may remove item 106(1) from the partitionedarea 202(1). The user 112 may then return the item 106(1) to the fixture102. Several users 112 may interact with the same fixture 102.

At a given time, an estimated location of a center of mass (COM) 204 maybe determined based on the weight data 122 provided by the load cells108(1)-(4) of the shelf 104. As the weight distribution on the shelf 104changes over time, so too does the estimated location of the COM 204change over time.

As described above, when the door 110 is opened air movement 116 mayresult in erroneous weight data 122. For example, the force of the airmovement 116 may press down disproportionately on a front edge of theshelf 104 that is closest to the door 110, and result in an estimatedlocation of COM 204 that is beyond the physical boundary of the shelf104.

In some implementations, the analysis module 126 may use the validitydata 128 to determine whether to use data obtained from other sensors136. For example, based on the door sensor data 124 image data obtainedfrom a camera sensor 136(1) may be disregarded for a period of time.

FIG. 3 depicts a graph 300 of weight data 122 and door sensor data 124that are used to determine when weight data 122 from load cells 108 isvalid, according to some implementations. In this illustration, thegraph 300 includes a horizontal axis in which time 302 increases fromleft to right wall of a perpendicular vertical axis indicates a weightvalue 304 obtained from the weight data 122. For example, the weightvalue 304 may comprise a value indicative of a weight on load cell 108at a particular time, the weight value 304 may comprise a sum of theload cells 108 at a particular shelf 104 a particular time, the weightvalue 304 may result from processing the weight data 122, and so forth.Also depicted is the configuration of the door 110 based on the doorsensor data 124, such as whether the door sensor 114 indicates the door110 is open or closed.

As time 302 progresses, the door 110 of the fixture 102 transitionsbetween a closed and an open configuration. At 306, the door is closed,followed by a door opening 308, the door being open 310, the doorclosing 312, and the door being closed 314. Over the span of time 302,the weight data 122 changes. For the example depicted here, assume thatno pick or place took place, and as a result, the weight of the items106 on the shelf 104 was unchanged. The weight values 304 show atemporary decrease during the period in which the door is opening 308,and show a temporary increase during the period in which the door isclosing 312. As described above, these changes in weight values 304result from air movement 116 that interacts with the shelf 104. Forexample, the movement of air resulting from the surface of the door 110displacing air may exert a pressure on the shelf 104, producing atemporary increase in weight values.

In some implementations, the analysis module 126 may be configured todesignate invalid time windows that include a transition from one doorconfiguration to another. For example, a first invalid time window316(1) is depicted that begins shortly before the door sensor data 124indicates a transition from a closed door 306 to an open door 310.Continuing the example, a second invalid time window 316(2) is depictedthat begins shortly before the door sensor data 124 indicates atransition from an open door 310 to the closed door 314. The invalidtime window 316 may have a duration that is manually configured,dynamically adjusted, and so forth. For example, the invalid time window316 may have a specified duration that extends from 100 ms before atransition in door configuration to a time 1000 ms after the transition.The analysis module 126 may generate validity data 128 that indicates asinvalid the weight values 304 or the weight data 122 upon which theweight values 304 are based. The invalid time window 316 allows time forthe weight data 122 or the weight values 304 to settle or otherwise stopbeing affected by air movements 116 or other effects resulting from achange in door configuration.

In some implementations, the invalid time window 316 may differ based onthe type of transition. For example, a first invalid time window 316(1)may be associated with the transition from a closed state to an openstate, while a second invalid time window 316(2) may be associated withthe transition from an open state to a close state. Continuing theexample, the first invalid time window 316(1) associated with atransition from closed to open may start at the time of the transitionand continue until 500 ms afterwards, while the second invalid timewindow 316(2) associated with a transition from open to closed mayextend from 700 milliseconds before a time of the transition until thetime of the transition.

In some implementations, the weight data 122 obtained during the invalidtime window 316 may be utilized, but may be flagged or otherwiseassociated with data indicative of a low confidence in the validity. Forexample, the weight data 122 may include data that is indicative of thedoor sensor data 124 or a change in the door state. This weight data 122may then be used by the inventory management system 120 to determineinteraction data 134, but the interaction data 134 may be subsequentlyassigned a low confidence value or other indicia indicative of lowconfidence due to the potentially invalid weight data 122.

Also depicted are valid time windows 318. The valid time windows 318depicted here are exclusive from the invalid time windows 316. Forexample, a first valid time window 318(1) is shown before the firstinvalid time window 316(1), a second valid time window 318(2) is shownbetween the end of the first invalid time window 316(1) and thebeginning of the second invalid time window 316(2), and a third validtime window 318(3) is depicted after the second invalid time window316(2). The analysis module 126 may generate validity data 128 thatindicates that the weight data 122 obtained during a valid time window318 is deemed to be valid. This valid weight data 122 may be usedgenerate interaction data 134.

In some implementations, the door sensor data 124 may be indicative ofthe extent to which the door 110 is open. The door sensor data 124 maybe indicative of an angular value or range or “bucket” value. Forexample, the door sensor data 124 may indicate that the door 110 is openwith an angle of 92°, that the door 110 is halfway open, or that thedoor 110 is slightly ajar. By utilizing door sensor data 124 thatprovides more information as to the configuration of the door 110, theanalysis module 126 may be better able to deal with dynamic changes inthe environment. For example, the door sensor data 124 may include ormay be used to generate a value that is indicative of how far the door110 is open. If this value exceeds a threshold value, the weight data122 may be deemed valid even though the door 110 is slightly open.Continuing the example, if the door 110 is slightly ajar, the weightdata 122 may still be considered valid and used to generate an inventorycount of items 106 at the shelf 104.

By utilizing the door sensor data 124 obtained from the door sensor 114,the analysis module 126 may be able to generate validity data 128, andmore accurately generate interaction data 134. For example, theinteraction data 134 may be more accurate because the erroneous weightdata 122 resulting from air movement 116 that occurs when the user 112opens or closes the door 110 may be disregarded.

The door sensor data 124 may be used in other ways. For example, if thedoor sensor data 124 indicates that the door 110 is closed, it may beassumed that the user 112 is unable to add or remove items 106 from theshelves 104. As a result, while the door sensor data 124 indicates thedoor 110 is closed, the analysis module 126 may deem the weight data 122during that time as being invalid, or as being of limited validity

FIG. 4 is a flow diagram 400 illustrating a process to determineinteraction data 134 based on weight data 122 that has been determinedto be valid, according to some implementations. The process may beimplemented at least in part by one or more of the fixture 102,computing devices located at or connected to the fixture 102, theinventory management system 120, and so forth.

At 402, weight data 122 generated by one or more load cells 108 at afixture 102 is accessed. For example, the weight data 122 may comprise atime series of weight values 304 generated by an array of load cells 108that weight a shelf 104.

At 404, door sensor data 124 obtained by one or more door sensors 114 isaccessed. For example, the door sensor 114 may comprise a switch, andthe door sensor data 124 may comprise an indication as to whether theswitch is open or closed at a particular time.

At 406, the door sensor data 124 is determined that is indicative of oneor more of a configuration of the door 110 at a particular time, changein the configuration of the door 110, and so forth. For example, thedoor sensor data 124 may be indicative of the door opening or closingfrom a first time to a second time. The door sensor data 124 maycomprise a single bit binary value with a zero indicating that the door110 is closed and one indicating that the door 110 is open. In someimplementations, the door sensor data 124 may be indicative of atransition at a particular time. For example, the door sensor data 124may indicate that the door 110 has changed between one configurationanother, but does not necessarily specify the start or endconfiguration.

In some implementations, the door sensor data 124 may be included withinthe weight data 122. One or more bits of the weight data 122 may bereplaced with values used to represent the door sensor data 124.Continuing the example, a least significant bit of the weight data 122may be replaced with single bit value that is indicative of the doorconfiguration.

The door sensor data 124 may be used to determine validity data 128indicative of validity of the weight data 122. For example, the validitydata 128 may be such that weight data 122 obtained while the door 110 isopening or closing is deemed to be invalid.

At 408, first weight data 122(1) which corresponds to a time between atleast the first time and the second time is determined to be invalid.For example, the first weight data 122(1) may comprise the weight data122 that occurs within the first invalid time window 316(1).

At 410, second weight data 122(2) which corresponds to a time before thefirst time or after the second time is determined to be valid. Forexample, the second weight data 122(2) may comprise weight data 122 thatproduces weight values 304 in one of the valid time windows 318illustrated in FIG. 3.

In some implementations, door sensor data 124 obtained from another door110 may be used to determine the validity data 128. For example, if theuser 112 opens a first door 110 the resulting air movement 116 mayaffect the load cells 108 of the fixtures 102 located to the left andright of that first door 110. The validity data 128 used to determinethe validity of weight data 122 may be based on the door sensor data 124associated with another door 110 on the same fixture 102, or from a door110 on another fixture 102.

At 412, interaction data 134 is generated using the second weight data122(2). The interaction data 134 may be indicative of one or more of apick or place of a type of item 106 to or from the shelf 104, or otherportion of the fixture 102. For example, the weight data 122 that hasvalidity data 128 indicating that it is deemed to be valid may be useddetermine a weight change at a particular shelf 104 or location thereonand produce interaction data 134 indicative of a pick or place of one ormore items 106.

In some implementations, at least a portion of the invalid weight datamay be used to determine the interaction data 134. For example, theweight data 122 associated with validity data 128 indicative of invaliddata may be used to generate interaction data 134 that has is deemed tobe of lower confidence or lesser reliability than when using weight data122 associated with validity data 128 indicative of valid data.

At 414, a quantity of the type of item 106 at the fixture 102 isdetermined. For example, the interaction data 134 may indicate that aquantity of six cans of frozen juice have been removed from the shelf104(1), and the quantity on hand may be decreased accordingly.

In some implementations, a computing device located at or near thefixture 102 may determine the validity data 128. To minimize networkbandwidth, minimize processing resources at the inventory managementsystem 120, or to provide other benefits the computing device may beconfigured to selectively transmit the weight data 122 that is deemed tobe valid to the inventory management system 120. For example, weightdata 122 deemed to be valid may be transmitted using a communicationinterface while weight data 122 deemed to be invalid is not. In otherimplementations, the fixture data 118 may be sent to the inventorymanagement system 120 which then makes a determination as to thevalidity as described above. For example, the inventory managementsystem 120 may disregard the weight data 122 for a particular intervalassociated with the door sensor data 124.

FIG. 5 depicts graphs 500 of weight values 304 from multiple shelves 104and a correspondence between those weight values 304 to determine whenweight data 122 from load cells 108 is valid, according to someimplementations. Shown are a weight graph 502 and a correspondence graph504.

The weight graph 502 depicts time 302 on the horizontal axis and weightvalues 304 along a vertical axis. In this weight graph 502, weight data122 from four shelves 104(1)-(4) are depicted. Similar to that shown inFIG. 3, several intervals of time 302 are labeled to show when the doorwas closed 306, when the door was opening 308, when the door was open310, when the door was closing 312, and when the door was closed 314. Aswith FIG. 3, for this illustration, assume that no items 106 were addedor removed. The decreases and increases in the weight values 304 are dueto something other than a pick or place, such as air movement 116, thevibration of the facility, and so forth.

The weight data 122 from the shelves 104 may comprise an overallaggregate weight value 304, separate sets of weight values 304associated with individual load cells 108, and so forth.

In some implementations, the door sensor 114 may be unavailable orinoperable. By using the weight data 122 from a plurality of shelves104, the analysis module 126 may determine if there is a correspondencebetween the weight data 122 from different shelves 104. The air movement116 or other environmental factors such as building vibration,earthquakes, and so forth, typically will affect more than one shelf 104at any given time. For example, the air movement 116 may result inchanges in the weight data 122 for several shelves 104 at about the sametime. It may be deemed extremely unlikely to have multiple users 112simultaneously interacting with several shelves 104. As a result,changes that occur at about the same time across multiple shelves 104may be deemed to be invalid.

The correspondence graph 504 indicates time 302 along the horizontalaxis while the vertical axis indicates a correlation coefficient 506.The correlation coefficient 506 comprises a value that is representativeof a correlation or other correspondence between two or more sets ofdata. For example, the correlation coefficient 506 may comprise theoutput of a correlation function that is using as input the weight data122 obtained from shelves 104(1)-104(4) at a particular time or during aparticular interval of time. The greater the value of the correlationcoefficient 506, the greater the similarity between the two or more setsof data that are being compared with one another. Depicted is acorrelation curve 508 that is indicative of the correlation between theweight data 122 from the four shelves 104(1)-(4) over time.

A correlation coefficient 506 that is greater that a correlationthreshold 510 indicates that the weight data 122 is deemed to besimilar. The greater the correlation coefficient 506, the greater thedegree of similarity. In this illustration, there are two peaks in whichthe correlation curve 508 exceeds the correlation threshold 510. Thesepeaks correspond to the door opening 308 and the door closing 312. Thefirst peak may be deemed a first invalid time window 512(1) while thesecond peak may be deemed a second invalid time window 512(2). Similarto that described above, the weight data 122 associated with the invalidtime window 512 may be disregarded. The weight data 122 that occursoutside of these invalid time windows 512 may be deemed to be valid andmay be further processed by the analysis module 126 to determineinteraction data 134.

FIG. 6 is a flow diagram 600 illustrating a process to determinevalidity of events based on weight data using a correspondence betweenweight data from multiple shelves, according to some implementations.The process may be implemented at least in part by one or more of thefixture 102, computing devices located at or connected to the fixture102, the inventory management system 120, and so forth.

At 602, first weight data 122(1) is obtained at a first shelf 104(1) ina fixture. For example, the first weight data 122(1) may be obtainedfrom a plurality of load cells 108 at the first shelf 104(1). The firstweight data 122(1) may be acquired during, or otherwise be associatedwith, a first time interval.

At 604, second weight data 122(2) is obtained at a second shelf 104(2)in a fixture. For example, the second weight data 122(2) may be obtainedfrom a plurality of load cells 108 at the second shelf 104(2). Thesecond weight data 122(2) may be acquired during, or otherwise beassociated with, the first time interval.

At 606, the correspondence value is determined based on thecorrespondence between the first weight data 122(1) and the secondweight data 122(2). The correspondence value may be determined forweight data 122 acquired from a plurality of shelves 104. For example,the correspondence value may comprise a value indicative of correlationbetween the first weight data 122(1) and the second weight data 122(2).While this example describes two shelves and first and second weightdata, it is understood that weight data from more than two shelves 104may be processed as described herein. In some implementations, across-correlation function may be used to generate the correspondencevalue. For example, the xcorr function provided in MATLAB by MathworksInc. of Natick, Mass. may be used. In another example, thecross-correlation function may comprise the scipy.signal.correlatefunction in the Scipy library as promulgated by scipy.org. Thecorrespondence value may be a single value, may be expressed as afunction, time series of values, matrix, and so forth. One or more ofthe sign or magnitude of the correspondence value may vary. For example,a positive correspondence value may be indicative of a significantcorrespondence between the first weight data 122(1) and the secondweight data 122(2). In another example, a negative correspondence valuemay be indicative of a significant correspondence. For ease ofillustration and not necessarily as a limitation, in this disclosure thecorrespondence value may be expressed as a positive number with greaternumbers indicating a greater degree of correspondence.

In some implementations, the first interval of time may be at least 250ms long, and the load cells 108 may acquire samples of weight data atleast 120 times per second. In other implementations, other durations ofthe first interval may be used, as well as other sample rates.

The correspondence value may be determined using weight data 122 fromshelves 104 that are immediately adjacent, such as one above another,comparing all of the shelves 104 on a rack, or other combinations orsubsets of adjacent shelves 104. For example, the correspondence valuemay be determined using weight data 122 from shelf 104(1) and shelf104(4). In one implementation, the physical layout data 132 or otherdata may be accessed to determine shelves 104 that are within athreshold distance of one another. For example, the correspondence valuemay be determined using weight data 122 from shelves 104 that are withinfive feet of a particular shelf 104, within five feet of a particulardoor 110, and so forth. The weight data 122 from the shelves that arewithin a threshold distance then be processed as described. In anotherexample, the correspondence value may be determined using weight data122 from shelves 104 that are within a common enclosure. For example, afreezer case may have six racks of shelves 104, each rack having fiveshelves. The correspondence value may be determined using weight data122 obtained from two or more of the thirty shelves in the freezer case.

In other implementations, other techniques may be used to determine thecorrespondence between the weight data 122 from different shelves 104.For example, the weight data 122(1) from a first shelf 104(1) may beprocessed to determine a first change in weight from a first time to asecond time. Continuing the example, the weight data 122(2) from asecond shelf 104(2) may be processed to determine a second change inweight from the first time to the second time. The correspondence valuemay thus correspond to one or more of the sign or magnitude of the firstchange in weight and the second change in weight.

At 608, validity data 128 is determined based on the correspondencevalue. As described above, it may be assumed that if adjacent shelves104 exhibit the same or similar changes with regard to their weight atthe same time, these changes may be due to something other than a pickor place. For example, the weight data 122 may be affected as a resultof the user 112 opening the door 110 and the subsequent air movement116. The validity data 128 may be determined for the weight data 122associated with one or more of the shelves 104. For example, if theweight data 122 is from shelf 104(1) and 104(4), the weight data 122 forboth may be designed as valid or invalid by the validity data 128. Inanother example, the weight data 122 from one shelf 104 may beassociated with the validity data 128 while the other is not.

In one implementation, the correspondence value may be compared with athreshold value. For example, if the correspondence value is greaterthan a threshold value, the first weight data 122(1) and the secondweight data 122(2) are deemed to be similar enough to one another duringthe first time interval to be deemed invalid. The validity data 128comprises information indicative of the validity of the first weightdata 122(1) and the second weight data 122(2).

In some implementations, processing may be performed to determine theoccurrence of an event may take place before the operations of 606. Forexample, the weight data 122 may be processed to determine an event,that is some change of the weight data 122. Responsive to the detectionof this event, the weight data 122 may be processed as described at 606to determine the correspondence value.

In some implementations, door status data may be determined based on thecorrespondence between the first weight data 122(1) and the secondweight data 122(2). For example, if the correspondence value exceeds athreshold amount, it may be deemed indicative of a transition in thedoor configuration.

At 610, interaction data 134 is generated using the weight datadesignated is valid by the validity data 128. For example, thecorrespondence value is less than threshold value, the first weight data122(1) and the second weight data 122(2) may be deemed to be valid. Forexample, the weight data 122 that has validity data 128 indicating thatit is deemed to be valid may be used determine a weight change at aparticular shelf 104 or location thereon and produce interaction data134 indicative of a pick or place of one or more items 106.

The analysis module 126 may assess the weight data 122 from adjacentshelves 104. Adjacent shelves 104 may include those shelves which areimmediately adjacent such as one above the other, immediately to theleft to right, or diagonally up to the left, diagonally down to theleft, and so forth. In some situations, adjacent shelves 104 may includeshelves 104 which are on the same rack or gondola, or in a rack orgondola that is nearby. In another implementation, adjacent shelves 104may include those shelves that are serviced by the same door 110. Forexample, several shelves 104 may be located on separate gondolas, butare accessed by a single door 110.

The analysis module 126 may use the validity data 128 to perform variousactions. For example, the validity data 128 may be transmitted to aserver for further use. Continuing the example, one or more of weightdata 122 from the shelves 104 associated with the validity data 128 maybe transmitted. In another example, the weight data 122 that isassociated with validity data 128 that is indicative of validity may besent to another device such as a server, stored, or otherwise processed.In yet another example, the weight data 122 that is associated withvalidity data 128 that is indicative of invalidity may be discarded orprocessed using one or more additional algorithms.

At 612, a quantity of the type of item at the fixture 102 is determined.For example, the interaction data 134 may indicate that a quantity ofsix cans of frozen juice have been removed from the shelf 104(1), andthe quantity on hand may be decreased accordingly.

FIG. 7 depicts a graph 700 of a two-dimensional space and a trajectoryof an estimated location of a center of mass of a shelf that may be usedto determine when weight data from load cells is valid, according tosome implementations.

As described above, air movement 116 or other phenomena may affect ashelf 104 in various ways. For example, as the door 110 is opened orclosed movement of air may result in pressure being applied to differentplaces on the shelf 104 over time. The weight data 122 may be assessedas described next to determine if the weight data is consistent withvalid or invalid data.

A two-dimensional space is depicted having a horizontal axis “x” 702that comprises values based on Expression 1 and a vertical axis “y” 704that is based on Expression 2.Weight_(LC2)+Weight_(LC3)−Weight_(LC1)  EXPRESSION 1Weight_(LC4)−Weight_(LC1)  EXPRESSION 2

In these expressions, the weights may comprise weight values 304, LC1indicates load cell 108(1), LC2 indicates load cell 108(2), LC3indicates load cell 108(3), and LC4 indicates load cell 108(4). In otherimplementations, other expressions may be utilized.

In this illustration, a trajectory 706 is depicted. The trajectory 706represents trajectory data that is indicative of a change within thespace over a period of time of the values produced by evaluatingExpressions 1 and 2. For example, the results of the expressions may beused to specify a point in the space at a particular time. For example,the trajectory 706 may be a visualization of a time series of data forthe values produced by evaluating Expressions 1 and 2. The trajectory706 does not necessarily correspond to movement in three-dimensionalphysical space, such as left, right, up, down, forward, and backward.

By utilizing simulation, testing, or other techniques, regions withinthe space may be designated as being consistent with data that is validor invalid. For example, a valid area 708 is depicted that correspondswith behavior within the space that is consistent with operation of theload cells 108 not impacted by the effects of air movement 116 and soforth. In comparison, an invalid region 710 within the space is depictedthat is consistent with a situation in which the load cells 108 on aparticular shelf 104 are producing invalid weight data 122. In asituation where there is erroneous weight data 122, the trajectory 706may stray into the invalid region 710.

In some implementations, machine learning techniques including, but notlimited to, classifiers, artificial neural networks, and so forth maytrained to recognize when the trajectory 706 is indicative of valid orinvalid data. For example, a machine learning module may be trainedusing fixture data 118 that includes weight data 122 and the door sensordata 124 obtained from the door sensor 114.

FIG. 8 depicts a graph 800 of a three-dimensional space and a trajectoryof an estimated location of a center of mass of a shelf that may be usedto determine when weight data from load cells is valid, according tosome implementations.

In some implementations, the analysis of the trajectory may be extendedinto a three-dimensional or greater space. A three-dimensional space isdepicted having a first axis “x” 802 that comprises values based onExpression 3, a second axis “y” 804 that is based on Expression 4, and athird axis “y” 806 that is based on Expression 5.Weight_(LC2)−Weight_(LC1)  EXPRESSION 3Weight_(LC3)−Weight_(LC1)  EXPRESSION 4Weight_(LC4)−Weight_(LC1)  EXPRESSION 5

In these expressions, the weights may comprise weight values 304, LC1indicates load cell 108(1), LC2 indicates load cell 108(2), LC3indicates load cell 108(3), and LC4 indicates load cell 108(4). In otherimplementations, other expressions may be utilized.

In this illustration, a trajectory 808 is depicted. The trajectory 808represents trajectory data that is indicative of a change within thespace over a period of time of the values produced by evaluatingExpressions 3, 4, and 5. For example, the results of the expressions maybe used to specify a point in the space at a particular time. Thetrajectory 808 does not necessarily correspond to movement inthree-dimensional physical space, such as left, right, up, down,forward, and backward.

Similar to that described above, an invalid region 810 is specified. Ifthe trajectory 808 strays within the invalid region 810, the weight data122 associated therewith may be deemed to be invalid.

By utilizing simulation, testing, or other techniques, regions withinthe space may be designated as being consistent with data that is validor invalid. For example, the invalid region 810 that is depictedcorresponds with behavior within the space that is consistent withoperation of the load cells 108 impact by the effects of air movement116 or other phenomena. In a situation where there is erroneous weightdata 122, the trajectory 808 may stray into the invalid region 810.

In some implementations, machine learning techniques including, but notlimited to, classifiers, artificial neural networks, and so forth maytrained to recognize when the trajectory 808 is indicative of valid orinvalid data. For example, a machine learning module may be trainedusing fixture data 118 that includes weight data 122 and the door sensordata 124 obtained from the door sensor 114.

In some implementations, the analysis of the trajectory 706 in thetwo-dimensional space and the trajectory 808 in the three-dimensionalspace may be used in conjunction with one another to determine thevalidity data 128. For example, if both trajectories intrude upon theinvalid region 710 and the invalid region 810 respectively, the weightdata 122 associated there with may be deemed to be invalid.

FIG. 9 is a flow diagram 900 illustrating a process that uses anestimated location of a center of mass 204 of a shelf to determine ifweight data is valid, according to some implementations. The process maybe implemented at least in part by one or more of the fixture 102,computing devices located at or connected to the fixture 102, theinventory management system 120, and so forth.

As described herein, the effects of phenomena such as air movement 116may result in changes to the weight data 122 produced by the load cells108 even in the absence of a pick or place of an item 106. As describedabove with regard to FIG. 2, the shelf 104 exhibits an estimatedlocation of a center of mass 204 that may be calculated using the weightdata 122 available at a particular time that was obtained from loadcells 108 of that particular shelf 104. The air movement 116 or otherphenomena may result in the estimated location of center of mass 204 todepart from its actual position and temporarily be located beyond thephysical boundary of the shelf. For example, a gust of air may impactthe front edge of the shelf 104, increasing the force measured by theload cells 108(2) and 108(3) that are located on the front of the shelf104. This increase may result in the estimated location of the center ofmass 204 shifting to a point that is several inches beyond the frontedge of the shelf 104. As the force of the impact by the air movement116 dissipates, the estimated location of the center of mass 204 mayreturn to approximately the same location as it was prior to the door110 opening.

At 902, weight data 122 is acquired from the load cells 108 at a shelf104 in the fixture 102.

At 904, a change in the weight data 122 that occurs during a timeinterval is determined. For example, an event detection module mayprocess the weight data 122.

At 906, an estimated location at the shelf 104 is determined using theweight data 122. For example, the estimated location may be indicativeof a center of mass, a location of a weight change, and so forth.

At 908, the estimated location is determined to be within a physicalboundary of the shelf 104. For example, the estimated location may besomewhere within the boundary of the edges of the shelf 104.

At 910, the weight data 122 is deemed to be valid. For example, validitydata 128 may be generated that is indicative of the weight data 122associated with the time interval being valid.

At 912, interaction data 134 is generated using the weight data 122. Theinteraction data 134 may be indicative of one or more of a pick or placeof a type of item to or from the shelf 104, or other portion of thefixture 102. For example, the weight data 122 that has validity data 128indicating that it is deemed to be valid may be used determine a weightchange at a particular shelf 104 or location thereon and produceinteraction data 134 indicative of a pick or place of one or more items106.

At 914, a quantity of the type of item at the fixture 102 is determined.For example, the interaction data 134 may indicate that a quantity ofsix cans of frozen juice have been added to the shelf 104(1), and thequantity on hand may be increased accordingly.

In the event that the estimated location is determined to be outside ofthe physical boundary of the shelf 104, the validity data 128 may beindicative of the weight data 122 being invalid.

By utilizing this process, the inventory management system 120 maydetermine the validity of the weight data 122 may act accordingly, suchas by utilizing valid weight data 122 and disregarding invalid weightdata 122.

In some implementations, door status data may be generated based on thisprocess. For example, the occurrence of the estimated location of thecenter of mass 204 at a location beyond the boundary of the shelf 104may be deemed to be indicative of a change in configuration of the door110.

Several of the techniques described herein may be combined. For example,the determination of the correspondence value may be combined with theprocess 900 to determine the validity data 128. Continuing the example,if within a specified interval of time the correspondence value exceedsa threshold value and the estimated location of the center of mass 204is outside of the physical boundary of the shelf 104, the weight data122 may be deemed to be invalid.

FIG. 10 is a flow diagram 1000 illustrating a process that uses atrajectory based on weight data 122 to determine if weight data isvalid, according to some implementations.

As described above with regard to FIGS. 7 and 8, trajectories may bedetermined in two-dimensional and three-dimensional spaces based on theweight data 122. These trajectories may be analyzed to determine if theweight data 122 is valid or invalid.

At 1002, weight data 122 is obtained from load cells 108 at a shelf 104in the fixture 102.

At 1004, event data based on the weight data 122 may be generated thatis indicative of an occurrence of an event between a first time and thesecond time.

At 1006, trajectory data is determined in a space using the weight data122 that occurred between the first time and second time. For example,one or more of the trajectory data represented by the trajectory 706 inthe two-dimensional space may be determined, the trajectory datarepresented by the trajectory 808 in the three-dimensional space bedetermined, or both.

As described above, in the two-dimensional space the trajectory data ofthe trajectory 706 may be calculated using Expressions 1 and 2 while inthe three-dimensional space the trajectory data represented bytrajectory 808 may be calculated using Expressions 1, 2, and 3.

At 1008, one or more points in the trajectory data that correspond to aninterval between the first time and the second time are determined tonot intersect a region within the space that is designated as invalid.

At 1010, the weight data 122 that occurs between the first time and thesecond time is designated as being valid.

In the event the points in the trajectory do intersect a region (such asan area or volume within the space) that is designated as invalid, theweight data 122 that occurs between the first time and second time isdesignated as being invalid.

At 1012, interaction data 134 is generated using the valid weight data122. The interaction data 134 may be indicative of one or more of a pickor place of a type of item to or from the shelf 104, or other portion ofthe fixture 102. For example, the weight data 122 that has validity data128 indicating that it is deemed to be valid may be used determine aweight change at a particular shelf 104 or location thereon and produceinteraction data 134 indicative of a pick or place of one or more items106.

At 1014, a quantity of the type of item at the fixture 102 isdetermined. For example, the interaction data 134 may indicate that aquantity of six bags of frozen peas have been added to the shelf 104(1),and the quantity on hand may be increased accordingly.

FIG. 11 is a block diagram 1100 of the analysis module 120 configured todetermine validity of weight data from load cells, according to someimplementations.

The load cells 108 generate weight data 122. An event detection module1102 may be used to generate event data 1104. One or more changedetection algorithms may be used to identify when the weight values havechanged in such a way as to meet or exceed a threshold value, such asspecified by threshold data 1106. For example, the event may bedetermined using a cumulative sum (CUSUM) function. An event may includeweight data 122 that begins at a start time and continues through an endtime.

The event data 1104 in some implementations may comprise a subset of theweight data 122. For example, the event data 1104 may comprise weightdata 122 that is been determined to include some change in values.

One or more door sensors 114 may generate door sensor data 124. The doorsensor data one and 24 may be provided to a door state determinationmodule 1110. The door state determination module 1110 is configured togenerate door status data 1112. The door status data 1112 is indicativeof one or more of the configuration of the door 110, the angle at whichthe door 110 is open, and so forth. For example, the door status data1112 may comprise a single bit value in which a zero is indicative ofthe door being closed and a one is indicative of the door being opened.In another example, door status data 1112 may indicate that the door isopen halfway, or at an angle of 105°, and so forth. In someimplementations, the door state determination module 1110 may beconfigured to determine a transition from one state to another, andgenerate door status data 1112 indicative of this transition. Forexample, the door state determination module 1110 may generate doorstate data 1112 that is indicative of the door transitioning from aclosed to an open state at a particular time.

The door state determination module 1110 may be configured to generatedoor state determination module 1110 using other techniques. Forexample, one or more of the techniques involving determination of thecorrespondence between weight data 122 from different shelves, theestimated location of center of mass, the trajectory determination, andso forth may be used to generate door status data 1112 in the absence ofa door sensor 114, or when a door sensor 114 is non-functional. Inimplementations in which the door status data 1112 is generated withoutthe use of door sensor data 124, the door status data 1112 may include aconfidence value or other data indicative of a likelihood that the doorstatus data 1112 accurately represents the situation of the door 110. Ifthe confidence value is less than a threshold value, the fixture data118 may be provided to a human operator to make a determination as tothe door status data 1112.

The door status data 1112 may then be used as described by the validitydetermination module 1108, or by other modules. For example, amaintenance module (not shown) may utilize the door status data 1112 todetermine if a particular door 110 has been left open for longer than athreshold amount of time, and dispatch an agent to close the door 110.

The event data 1104 may be processed by a validity determination module1108. The validity determination module 1108 may utilize one or more ofthe techniques or processes described herein to generate the validitydata 128, either alone or in various combinations. The validitydetermination module 108 may access one or more threshold valuesspecified by the threshold data 1106.

In one implementation, the door status data 1112 may be provided to thevalidity determination module 1108. For example, as described above withregard to FIGS. 3 and 4, the door status data 1112 may be used toindicate if the door 110 is open or close, and validity data 128 may bedetermined based at least in part on the door status data 1112.

In another implementation, the validity determination module 1108 may beconfigured to generate correspondence value using the weight data 122from a plurality of the shelves. In yet another implementation, thevalidity determination module 1108 may be configured to determine theestimated center of mass 204 is outside of a physical boundary of theshelf. In still another implementation, the validity determinationmodule 1108 may generate one or more multidimensional trajectories basedon the weight data 122.

The validity determination module 1108 may provide as output validweight data 1114. The valid weight data 1114 may comprise the event data1104 or the weight data 122 that is associated with validity data 128indicative of valid data. For example, valid weight data 1114 maycomprise weight data obtained during the valid time window 318.

An interaction determination module 1116 may use a valid weight data1114 as input. The interaction determination module 116 may access oneor more thresholds specified by the threshold data 1106. The interactiondetermination module 116 may access one or more of the item data 130,the physical layout data 132, and so forth during operation. Theinteraction determination module 1116 may use the valid weight data 1114to determine the fixture 102, shelf 104, lane within the shelf, and soforth associated with a weight change resulting from a pick or place ofone or more items 106. As described above, the item data 130 providesinformation such as the weight of an individual item, the physicallayout data 132 associates a particular lane or location on the shelf104 with that particular item 106.

By using the valid weight data 1114 to determine a change in weight anda lane or other location on the shelf 104 so she with that change inweight, the interaction data 134 may be generated. For example, theinteraction data 134 may indicate that a quantity of one can of frozenorange juice was removed from shelf 104(1) at a particular time. Byutilizing the techniques described herein to determine the validity data128 and by avoiding the use of invalid weight data 122, overallperformance of the system is improved and the system may operate moreefficiently.

Furthermore, by utilizing the valid weight data 1114, the system mayminimize or avoid altogether workflows that involves sending at least aportion of the data obtained from one or more of the sensors in thefacility to a human operator or a more computationally intensiveprocessing system. Without the validity determination module 1108,events that have been deemed to have a low confidence, or theinteraction data 134 based on that low confidence weight data 122, beprovided to a human operator who then makes a determination as to whatactually occurred at the facility. By using the techniques describedherein, this manual processing by human operator may be significantlyreduced or limited altogether.

FIG. 12 is a block diagram 1200 illustrating a materials handlingfacility (facility) 1202 using the system 100, according to someimplementations. A facility 1202 comprises one or more physicalstructures or areas within which one or more items 106(1), 108(2), . . ., 108(Q) may be held. As used in this disclosure, letters in parenthesissuch as “(Q)” indicate an integer value greater than or equal to zero.The items 106 may comprise physical goods, such as books,pharmaceuticals, repair parts, electronic gear, and so forth.

The facility 1202 may include one or more areas designated for differentfunctions with regard to inventory handling. In this illustration, thefacility 1202 includes a receiving area 1204, a storage area 1206, and atransition area 1208.

The receiving area 1204 may be configured to accept items 106, such asfrom suppliers, for intake into the facility 1202. For example, thereceiving area 1204 may include a loading dock at which trucks or otherfreight conveyances unload the items 106. In some implementations, theitems 106 may be processed, such as at the receiving area 1204, togenerate at least a portion of the item data 130. For example, an item106 may be imaged or otherwise scanned to develop reference images orrepresentations of the item 106 at the receiving area 1204.

The storage area 1206 is configured to store the items 106. The storagearea 1206 may be arranged in various physical configurations. In oneimplementation, the storage area 1206 may include one or more aisles1210. The aisle 1210 may be configured with, or defined by, fixtures 102on one or both sides of the aisle 1210. The fixtures 102 may include oneor more of a fixture 102, a rack, a case, a cabinet, a bin, a floorlocation, or other suitable storage mechanisms for holding, supporting,or storing the items 106. For example, the fixtures 102 may compriseshelves 102 with partitioned areas 202 designated therein. The fixtures102 may be affixed to the floor or another portion of the structure ofthe facility 1202. The fixtures 102 may also be movable such that thearrangements of aisles 1210 may be reconfigurable. In someimplementations, the fixtures 102 may be configured to moveindependently of an outside operator. For example, the fixtures 102 maycomprise a rack with a power source and a motor, operable by a computingdevice to allow the rack to move from one location within the facility1202 to another.

One or more users 112(1), 110(2), . . . , 110(U) and totes 1212(1),1214(2), . . . , 1214(T) or other material handling apparatus may movewithin the facility 1202. For example, the user 112 may move aboutwithin the facility 1202 to pick or place the items 106 in variousfixtures 102, placing them on the tote 1212 for ease of transport. Thetote 1212 is configured to carry or otherwise transport one or moreitems 106. For example, the tote 1212 may include a basket, cart, bag,bin, and so forth. In other implementations, other material handlingapparatuses such as robots, forklifts, cranes, aerial drones, and soforth, may move about the facility 1202 picking, placing, or otherwisemoving the items 106. For example, a robot may pick an item 106 from afirst fixture 102(1) and move the item 106 to a second fixture 102(2).

One or more sensors 136 may be configured to acquire information in thefacility 1202. The sensors 136 may include, but are not limited to,camera 136(1), depth sensors 136(2), load cells 108, optical sensorarrays 124(13), proximity sensors 136(6), and so forth. The sensors 136may be stationary or mobile, relative to the facility 1202. For example,the fixtures 102 may contain load cells 108 to generate weight signals1202, cameras 136(1) to acquire images of picking or placement of items106 on shelves 102, optical sensor arrays 124(13) to detect shadows ofthe user's 110 hands at the fixtures 102, and so forth. In anotherexample, the facility 1202 may include a camera 136(1) to obtain imagesof the user 112 or other objects in the facility 1202. The sensors 136are discussed in more detail below with regard to FIG. 13.

While the storage area 1206 is depicted as having one or more aisles1210, fixtures 102 storing the items 106, sensors 136, and so forth, itis understood that the receiving area 1204, the transition area 1208, orother areas of the facility 1202 may be similarly equipped. Furthermore,the arrangement of the various areas within the facility 1202 isdepicted functionally rather than schematically. For example, in someimplementations, multiple different receiving areas 1204, storage areas1206, and transition areas 1208 may be interspersed rather thansegregated in the facility 1202.

The facility 1202 may include, or be coupled to, an inventory managementsystem 120. The inventory management system 120 is configured tointeract with users 112 or devices such as sensors 136, robots, materialhandling equipment, computing devices, and so forth, in one or more ofthe receiving area 1204, the storage area 1206, or the transition area1208.

During operation of the facility 1202, the sensors 136 may be configuredto provide sensor data, or information based on the sensor data, to theinventory management system 120. The sensor data may include image data,non-image data, weight data 122 obtained from load cells 108, and soforth. The sensors 136 are described in more detail below with regard toFIG. 13.

The inventory management system 120 or other systems may use the sensordata to track the location of objects within the facility 1202, movementof the objects, or provide other functionality. Objects may include, butare not limited to, items 106, users 112, totes 1212, and so forth. Forexample, a series of images acquired by the camera 136(1) may indicateremoval by the user 112 of an item 106 from a particular location on thefixture 102 and placement of the item 106 on or at least partiallywithin the tote 1212.

The facility 1202 may be configured to receive different kinds of items106 from various suppliers and to store them until a customer orders orretrieves one or more of the items 106. A general flow of items 106through the facility 1202 is indicated by the arrows of FIG. 12.Specifically, as illustrated in this example, items 106 may be receivedfrom one or more suppliers, such as manufacturers, distributors,wholesalers, and so forth, at the receiving area 1204. In variousimplementations, the items 106 may include merchandise, commodities,perishables, or any suitable type of item 106, depending on the natureof the enterprise that operates the facility 1202.

Upon being received from a supplier at the receiving area 1204, theitems 106 may be prepared for storage in the storage area 1206. Forexample, in some implementations, items 106 may be unpacked or otherwiserearranged. The inventory management system 120 may include one or moresoftware applications executing on a computer system to provideinventory management functions. These inventory management functions mayinclude maintaining information indicative of the type, quantity,condition, cost, location, weight, or any other suitable parameters withrespect to the items 106. The items 106 may be stocked, managed, ordispensed in terms of countable units, individual units, or multipleunits, such as packages, cartons, crates, pallets, or other suitableaggregations. Alternatively, some items 106, such as bulk products,commodities, and so forth, may be stored in continuous or arbitrarilydivisible amounts that may not be inherently organized into countableunits. Such items 106 may be managed in terms of a measurable quantitysuch as units of length, area, volume, weight, time, duration, or otherdimensional properties characterized by units of measurement. Generallyspeaking, a quantity of an item 106 may refer to either a countablenumber of individual or aggregate units of an item 106 or a measurableamount of an item 106, as appropriate.

After arriving through the receiving area 1204, items 106 may be storedwithin the storage area 1206. In some implementations, like items 106may be stored or displayed together in the fixtures 102 such as in bins,on shelves 102, hanging from pegboards, and so forth. In thisimplementation, all items 106 of a given kind are stored in one fixture102. In other implementations, like items 106 may be stored in differentfixtures 102. For example, to optimize retrieval of certain items 106having frequent turnover within a large physical facility 1202, thoseitems 106 may be stored in several different fixtures 102 to reducecongestion that might occur at a single fixture 102.

When a customer order specifying one or more items 106 is received, oras a user 112 progresses through the facility 1202, the correspondingitems 106 may be selected or “picked” from the fixtures 102 containingthose items 106. In various implementations, item picking may range frommanual to completely automated picking. For example, in oneimplementation, a user 112 may have a list of items 106 they desire andmay progress through the facility 1202 picking items 106 from fixtures102 within the storage area 1206 and placing those items 106 into a tote1212. In other implementations, employees of the facility 1202 may pickitems 106 using written or electronic pick lists derived from customerorders. These picked items 106 may be placed into the tote 1212 as theemployee progresses through the facility 1202.

After items 106 have been picked, the items 106 may be processed at atransition area 1208. The transition area 1208 may be any designatedarea within the facility 1202 where items 106 are transitioned from onelocation to another or from one entity to another. For example, thetransition area 1208 may be a packing station within the facility 1202.When the item 106 arrives at the transition area 1208, the items 106 maybe transitioned from the storage area 1206 to the packing station.Information about the transition may be maintained by the inventorymanagement system 120.

In another example, if the items 106 are departing the facility 1202, alist of the items 106 may be obtained and used by the inventorymanagement system 120 to transition responsibility for, or custody of,the items 106 from the facility 1202 to another entity. For example, acarrier may accept the items 106 for transport with that carrieraccepting responsibility for the items 106 indicated in the list. Inanother example, a user 112 may purchase or rent the items 106 andremove the items 106 from the facility 1202. During use of the facility1202, the user 112 may move about the facility 1202 to perform varioustasks, such as picking or placing the items 106 in the fixtures 102.

To facilitate operation of the facility 1202, the inventory managementsystem 120 is configured to use the sensor data including the weightdata 122 and other information such as the item data 130, the physicallayout data 132, the non-weight data 1430, and so forth, to generateinteraction data 134.

The interaction data 134 may provide information about an interaction,such as a pick of an item 106 from the fixture 102, a place of an item106 to the fixture 102, a touch made to an item 106 at the fixture 102,a gesture associated with an item 106 at the fixture 102, and so forth.The interaction data 134 may include one or more of the type ofinteraction, interaction location identifier indicative of where fromthe fixture 102 the interaction took place, item identifier, quantitychange to the item 106, user identifier, and so forth. The interactiondata 134 may then be used to further update the item data 130. Forexample, the quantity of items 106 on hand at a particular partitionedarea 202 on the fixture 102 may be changed based on an interaction thatpicks or places one or more items 106.

The inventory management system 120 may combine or otherwise utilizedata from different sensors 136 of different types, including the loadcells 108. For example, weight data 122 obtained from load cells 108 atthe fixture 102 may be used in conjunction with non-weight data 1430such as the image data to determine the interaction data 134.

FIG. 13 is a block diagram 1300 illustrating additional details of thefacility 1202, according to some implementations. The facility 1202 maybe connected to one or more networks 1302, which in turn connect to oneor more servers 1304. The network 1302 may include private networks suchas an institutional or personal intranet, public networks such as theInternet, or a combination thereof. The network 1302 may utilize wiredtechnologies (e.g., wires, fiber optic cables, and so forth), wirelesstechnologies (e.g., radio frequency, infrared, acoustic, optical, and soforth), or other connection technologies. The network 1302 isrepresentative of any type of communication network, including one ormore of data networks or voice networks. The network 1302 may beimplemented using wired infrastructure (e.g., copper cable, fiber opticcable, and so forth), a wireless infrastructure (e.g., cellular,microwave, satellite, and so forth), or other connection technologies.

The servers 1304 may be configured to execute one or more modules orsoftware applications associated with the inventory management system120 or other systems. While the servers 1304 are illustrated as being ina location outside of the facility 1202, in other implementations, atleast a portion of the servers 1304 may be located at the facility 1202.The servers 1304 are discussed in more detail below with regard to FIG.14.

The users 112, the totes 1212, or other objects in the facility 1202 maybe equipped with one or more tags 1306. The tags 1306 may be configuredto emit a signal 1308. In one implementation, the tag 1306 may be aradio frequency identification (RFID) tag 1306 configured to emit a RFsignal 1308 upon activation by an external signal. For example, theexternal signal may comprise a radio frequency signal or a magneticfield configured to energize or activate the RFID tag 1306. In anotherimplementation, the tag 1306 may comprise a transmitter and a powersource configured to power the transmitter. For example, the tag 1306may comprise a Bluetooth® Low Energy (BLE) transmitter and battery. Inother implementations, the tag 1306 may use other techniques to indicatepresence of the tag 1306. For example, an acoustic tag 1306 may beconfigured to generate an ultrasonic signal 1308, which is detected bycorresponding acoustic receivers. In yet another implementation, the tag1306 may be configured to emit an optical signal 1308.

The inventory management system 120 may be configured to use the tags1306 for one or more of identification of the object, determining alocation of the object, and so forth. For example, the users 112 maywear tags 1306, the totes 1212 may have tags 1306 affixed, and so forth,which may be read and, based at least in part on signal strength, usedto determine identity and location.

Generally, the inventory management system 120 or other systemsassociated with the facility 1202 may include any number and combinationof input components, output components, and servers 1304.

The one or more sensors 136 (including the load cells 108) may bearranged at one or more locations within the facility 1202. For example,the sensors 136 may be mounted on or within a floor, wall, at a ceiling,at a fixture 102, on a tote 1212, may be carried or worn by a user 112,and so forth.

The sensors 136 may include one or more load cells 108. One or more loadcells 108 are configured to measure the weight of a load, such as theitem 106, the tote 1212, or other objects. The load cells 108 may beconfigured to measure the weight of the load at one or more of thefixtures 102, the tote 1212, on the floor of the facility 1202, and soforth. For example, the fixture 102 may include a plurality ofpartitioned areas 202 or platforms, with one or more load cells 108beneath each one to provide weight signals 1202 about an individualpartitioned area 202 or platform. The load cells 108 may include one ormore sensing mechanisms to determine the weight of a load. These sensingmechanisms may include piezoresistive devices, piezoelectric devices,capacitive devices, electromagnetic devices, optical devices,potentiometric devices, microelectromechanical devices, and so forth.The sensing mechanisms of load cells 108 may operate as transducers thatgenerate one or more signals based on an applied force, such as that ofthe load due to gravity. For example, the load cell 108 may comprise aload cell having a strain gauge and a structural member that deformsslightly when weight is applied. By measuring a change in the electricalcharacteristic of the strain gauge, such as capacitance or resistance,the weight may be determined. In another example, the load cell 108 maycomprise a force sensing resistor (FSR). The FSR may comprise aresilient material that changes one or more electrical characteristicswhen compressed. For example, the electrical resistance of a particularportion of the FSR may decrease as the particular portion is compressed.The inventory management system 120 may use the data acquired by theload cells 108 to identify an object, determine a change in the quantityof objects, determine a location of an object, maintain shippingrecords, and so forth.

The sensors 136 may include one or more cameras 136(1) or other imagingsensors. The one or more cameras 136(1) may include imaging sensorsconfigured to acquire images of a scene. The cameras 136(1) areconfigured to detect light in one or more wavelengths including, but notlimited to, terahertz, infrared, visible, ultraviolet, and so forth. Theone or more cameras 136(1) may comprise charge coupled devices (CCD),complementary metal oxide semiconductor (CMOS) devices, microbolometers,and so forth. The inventory management system 120 may use image dataacquired by the one or more cameras 136(1) during operation of thefacility 1202. For example, the inventory management system 120 mayidentify items 106, users 112, totes 1212, and so forth, based at leastin part on their appearance within the image data acquired by the one ormore cameras 136(1). The one or more cameras 136(1) may be mounted invarious locations within the facility 1202. For example, a camera 136(1)may be mounted overhead, on fixtures 102, may be worn or carried byusers 112, may be affixed to totes 1212, and so forth.

One or more depth sensors 136(2) may also be included in the sensors136. The depth sensors 136(2) are configured to acquire spatial orthree-dimensional (3D) data, such as depth information, about objectswithin a field of view. The depth sensors 136(2) may include rangecameras, lidar systems, sonar systems, radar systems, structured lightsystems, stereo vision systems, optical interferometry systems, and soforth. The inventory management system 120 may use the 12D data acquiredby the depth sensors 136(2) to identify objects, determine a location ofan object in 12D real space, and so forth. One or more buttons 124(3)may be configured to accept input from the user 112. The buttons 124(3)may comprise mechanical, capacitive, optical, or other mechanisms. Forexample, the buttons 124(3) may comprise mechanical switches configuredto accept an applied force from a touch of the user 112 to generate aninput signal. The inventory management system 120 may use data from thebuttons 124(3) to receive information from the user 112. For example,the tote 1212 may be configured with a button 124(3) to accept inputfrom the user 112 and send information indicative of the input to theinventory management system 120.

The sensors 136 may include one or more touch sensors 136(4). The touchsensors 136(4) may use resistive, capacitive, surface capacitance,projected capacitance, mutual capacitance, optical, InterpolatingForce-Sensitive Resistance (IFSR), or other mechanisms to determine theposition of a touch or near-touch. For example, the IFSR may comprise amaterial configured to change electrical resistance responsive to anapplied force. The location within the material of that change inelectrical resistance may indicate the position of the touch. Theinventory management system 120 may use data from the touch sensors136(4) to receive information from the user 112. For example, the touchsensor 136(4) may be integrated with the tote 1212 to provide atouchscreen with which the user 112 may select from a menu one or moreparticular items 106 for picking, enter a manual count of items 106 at afixture 102, and so forth.

One or more microphones 124(5) may be configured to acquire informationindicative of sound present in the environment. In some implementations,arrays of microphones 124(5) may be used. These arrays may implementbeamforming techniques to provide for directionality of gain. Theinventory management system 120 may use the one or more microphones124(5) to acquire information from acoustic tags 1306, accept voiceinput from the users 112, determine ambient noise level, and so forth.

The sensors 136 may include proximity sensors 136(6) used to determinepresence of an object, such as the user 112, the tote 1212, and soforth. The proximity sensors 136(6) may use optical, electrical,ultrasonic, electromagnetic, or other techniques to determine a presenceof an object. In some implementations, the proximity sensors 136(6) mayuse an optical emitter and an optical detector to determine proximity.For example, an optical emitter may emit light, a portion of which maythen be reflected by the object back to the optical detector to providean indication that the object is proximate to the proximity sensor136(6). In other implementations, the proximity sensors 136(6) maycomprise a capacitive proximity sensor 136(6) configured to provide anelectrical field and determine a change in electrical capacitance due topresence or absence of an object within the electrical field.

The proximity sensors 136(6) may be configured to provide sensor dataindicative of one or more of a presence or absence of an object, adistance to the object, or characteristics of the object. An opticalproximity sensor 136(6) may use time-of-flight (ToF), structured light,interferometry, or other techniques to generate the distance data. Forexample, ToF determines a propagation time (or “round-trip” time) of apulse of emitted light from an optical emitter or illuminator that isreflected or otherwise returned to an optical detector. By dividing thepropagation time in half and multiplying the result by the speed oflight in air, the distance to an object may be determined. In anotherimplementation, a structured light pattern may be provided by theoptical emitter. A portion of the structured light pattern may then bedetected on the object using a sensor 136 such as a camera 136(1). Basedon an apparent distance between the features of the structured lightpattern, the distance to the object may be calculated. Other techniquesmay also be used to determine distance to the object. In anotherexample, the color of the reflected light may be used to characterizethe object, such as skin, clothing, tote 1212, and so forth.

The sensors 136 may include one or more optical sensors 136(7). Theoptical sensors 136(7) may be configured to provide data indicative ofone or more of color or intensity of light impinging thereupon. Forexample, the optical sensor 136(7) may comprise a photodiode andassociated circuitry configured to generate a signal or data indicativeof an incident flux of photons. As described below, the optical sensorarray 124(13) may comprise a plurality of the optical sensors 136(7).For example, the optical sensor 136(7) may comprise an array of ambientlight sensors such as the ISL76683 as provided by Intersil Corporationof Milpitas, Calif., USA, or the MAX44009 as provided by MaximIntegrated of San Jose, Calif., USA. In other implementations, otheroptical sensors 136(7) may be used. The optical sensors 136(7) may besensitive to one or more of infrared light, visible light, orultraviolet light. For example, the optical sensors 136(7) may besensitive to infrared light, and infrared light sources such as LEDs mayprovide illumination.

The optical sensors 136(7) may include photodiodes, photoresistors,photovoltaic cells, quantum dot photoconductors, bolometers,pyroelectric infrared detectors, and so forth. For example, the opticalsensor 136(7) may use germanium photodiodes to detect infrared light.

One or more radio frequency identification (RFID) readers 124(8), nearfield communication (NFC) systems, and so forth, may be included assensors 136. For example, the RFID readers 124(8) may be configured toread the RF tags 1306. Information acquired by the RFID reader 124(8)may be used by the inventory management system 120 to identify an objectassociated with the RF tag 1306 such as the item 106, the user 112, thetote 1212, and so forth. For example, based on information from the RFIDreaders 124(8) detecting the RF tag 1306 at different times and RFIDreaders 124(8) having different locations in the facility 1202, avelocity of the RF tag 1306 may be determined.

One or more RF receivers 124(9) may also be included as sensors 136. Insome implementations, the RF receivers 124(9) may be part of transceiverassemblies. The RF receivers 124(9) may be configured to acquire RFsignals 1308 associated with Wi-Fi®, Bluetooth®, ZigBee®, 2G, 12G, 13G,LTE, or other wireless data transmission technologies. In someimplementations, the RF receivers 124(9) may detect signals transmittedat frequencies such as below 15 MHz. The RF receivers 124(9) may provideinformation associated with data transmitted via radio frequencies,signal strength of RF signals 1308, and so forth. For example,information from the RF receivers 124(9) may be used by the inventorymanagement system 120 to determine a location of an RF source, such as acommunication interface onboard the tote 1212.

The sensors 136 may include one or more accelerometers 124(10), whichmay be worn or carried by the user 112, mounted to the tote 1212, and soforth. The accelerometers 124(10) may provide information such as thedirection and magnitude of an imposed acceleration. Data such as rate ofacceleration, determination of changes in direction, speed, and soforth, may be determined using the accelerometers 124(10).

A gyroscope 124(11) may provide information indicative of rotation of anobject affixed thereto. For example, the tote 1212 or other objects maybe equipped with a gyroscope 124(11) to provide data indicative of achange in orientation of the object.

A magnetometer 124(12) may be used to determine an orientation bymeasuring ambient magnetic fields, such as the terrestrial magneticfield. The magnetometer 124(12) may be worn or carried by the user 112,mounted to the tote 1212, and so forth. For example, the magnetometer124(12) mounted to the tote 1212 may act as a compass and provideinformation indicative of which direction the tote 1212 is oriented.

An optical sensor array 124(13) may comprise one or optical sensors136(7). The optical sensors 136(7) may be arranged in a regular,repeating, or periodic two-dimensional arrangement such as a grid. Theoptical sensor array 124(13) may generate image data. For example, theoptical sensor array 124(13) may be arranged within or below a fixture102 and obtain information about shadows of items 106, hand of the user112, and so forth.

The sensors 136 may include other sensors 136(S) as well. For example,the other sensors 136(S) may include light curtains, ultrasonicrangefinders, thermometers, barometric sensors, hygrometers, and soforth. For example, the inventory management system 120 may useinformation acquired from thermometers and hygrometers in the facility1202 to direct the user 112 to check on delicate items 106 stored in aparticular fixture 102, which is overheating, too dry, too damp, and soforth.

In one implementation, a light curtain may utilize a linear array oflight emitters and a corresponding linear array of light detectors. Forexample, the light emitters may comprise a line of infrared lightemitting diodes (LEDs) or vertical cavity surface emitting lasers(VCSELs) that are arranged above a top fixture 102 in front of thefixture 102, while the light detectors comprise a line of photodiodessensitive to infrared light arranged below the light emitters. The lightemitters produce a “lightplane” or sheet of infrared light that is thendetected by the light detectors. An object passing through thelightplane may decrease the amount of light falling upon the lightdetectors. For example, the user's 110 hand would prevent at least someof the light from light emitters from reaching a corresponding lightdetector. As a result, a position along the linear array of the objectmay be determined that is indicative of a touchpoint. This position maybe expressed as touchpoint data, with the touchpoint being indicative ofthe intersection between the hand of the user 112 and the sheet ofinfrared light. In some implementations, a pair of light curtains may bearranged at right angles relative to one another to providetwo-dimensional touchpoint data indicative of a position of touch in aplane. Input from the light curtain, such as indicating occlusion from ahand of a user 112 may be used to trigger acquisition or selection ofimage data for processing by the analysis module 126.

The sensors 136 may also include an instrumented auto-facing unit (AFU)136(14). The instrumented AFU 136(14) may comprise a position sensorconfigured to provide data indicative of displacement of a pusher. As anitem 106 is removed from the AFU, the pusher moves, such as under theinfluence of a spring, and pushes the remaining items 106 in the AFU tothe front of the fixture 102. By using data from the position sensor,and given item data 130 such as a depth of an individual item 106, acount may be determined, based on a change in position data. Forexample, if each item 106 is 1 inch deep, and the position dataindicates a change of 13 inches, the quantity held by the AFU may havechanged by 13 items 106. This count information may be used to confirmor provide a cross check for a count obtained by other means, such asanalysis of the image data, weight data 122, and so forth.

The sensors 136 may also include the door sensor 114. In someimplementations, the other sensors described herein may be used togenerate door sensor data 124. For example, the proximity sensor 136(6)mounted on the shelf 104 may generate door sensor data 124 based on thedetection of proximity of the door 110. In another example,accelerometer 136(10) may be mounted to the door 110 and is used togenerate door sensor data 124 indicative of the motion of the door 110as the door 110 is opened or closed.

In some implementations, the door sensor 114 may comprise one or more ofa switch, a capacitive proximity sensor, an ultrasonic proximity sensor,an optical proximity sensor, a magnetic field sensor such as themagnetometer 136(12) or a magnetic reed switch, an anemometer, atemperature sensor, a potentiometer, a force sensitive resistor, anoptical encoder, a camera, and so forth. The magnetic field sensor maybe mounted doorframe on the fixture 102 and a magnet be located withinthe door 110. Detection of the magnetic field produced by the magnet maybe used to generate the door sensor data 124. The anemometer may beconfigured to detect the air movement 116, which is then used togenerate the door sensor data 124. The temperature sensor may comprise athermocouple that detects a change in temperature due to the door 110being opened or closed, and this change in temperature may then be usedto generate door sensor data 124. A potentiometer may be mounted on thehinge mechanism to generate door sensor data 124 that indicates a changein electrical resistance that is indicative of the angle of the door 110with respect to the fixture 102. The force sensitive resistor may beused to detect the pressure of the door 110 against the doorframe of thefixture 102, or a portion of a hinge mechanism. An optical encoder maycomprise an optical target and an optical transmitter and receiver thatread the optical target. The optical encoder may be used to determinewhether the doors open or closed, in some implementations may be used toindicate the angle at which the door is open. A camera may be used togenerate image data that is then processed to determine theconfiguration of the door 110. In some implementations, a plurality ofdoor sensors 114 may be used simultaneously on the same fixture 102.

In some implementations, the camera 136(1) or other sensors 136(5) mayinclude hardware processors, memory, and other elements configured toperform various functions. For example, the sensor 136(1) may beconfigured to generate image data, send the image data to another devicesuch as the server 1304, and so forth.

The facility 1202 may include one or more access points 1310 configuredto establish one or more wireless networks. The access points 1310 mayuse Wi-Fi®, NFC, Bluetooth®, or other technologies to establish wirelesscommunications between a device and the network 1302. The wirelessnetworks allow the devices to communicate with one or more of thesensors 136, the inventory management system 120, the optical sensorarrays 124(13), the tag 1306, a communication device of the tote 1212,or other devices.

Output devices 1312 may also be provided in the facility 1202. Theoutput devices 1312 are configured to generate signals, which may beperceived by the user 112 or detected by the sensors 136. In someimplementations, the output devices 1312 may be used to provideillumination of the optical sensor array 124(13).

Haptic output devices 1312(1) are configured to provide a signal thatresults in a tactile sensation to the user 112. The haptic outputdevices 1312(1) may use one or more mechanisms such as electricalstimulation or mechanical displacement to provide the signal. Forexample, the haptic output devices 1312(1) may be configured to generatea modulated electrical signal, which produces an apparent tactilesensation in one or more fingers of the user 112. In another example,the haptic output devices 1312(1) may comprise piezoelectric or rotarymotor devices configured to provide a vibration, which may be felt bythe user 112.

One or more audio output devices 1312(2) may be configured to provideacoustic output. The acoustic output includes one or more of infrasonicsound, audible sound, or ultrasonic sound. The audio output devices1312(2) may use one or more mechanisms to generate the acoustic output.These mechanisms may include, but are not limited to, the following:voice coils, piezoelectric elements, magnetorestrictive elements,electrostatic elements, and so forth. For example, a piezoelectricbuzzer or a speaker may be used to provide acoustic output.

The display devices 1312(3) may be configured to provide output, whichmay be seen by the user 112 or detected by a light-sensitive sensor suchas a camera 136(1) or an optical sensor 136(7). In some implementations,the display devices 1312(3) may be configured to produce output in oneor more of infrared, visible, or ultraviolet light. The output may bemonochrome or in color. The display devices 1312(3) may be one or moreof emissive, reflective, microelectromechanical, and so forth. Anemissive display device 1312(3), such as using LEDs, is configured toemit light during operation. In comparison, a reflective display device1312(3), such as using an electrophoretic element, relies on ambientlight to present an image. Backlights or front lights may be used toilluminate non-emissive display devices 1312(3) to provide visibility ofthe output in conditions where the ambient light levels are low.

The display devices 1312(3) may be located at various points within thefacility 1202. For example, the addressable displays may be located onfixtures 102, totes 1212, on the floor of the facility 1202, and soforth.

Other output devices 1312(P) may also be present. For example, the otheroutput devices 1312(P) may include scent/odor dispensers, documentprinters, 12D printers or fabrication equipment, and so forth.

FIG. 14 illustrates a block diagram 1400 of a server 1304 configured tosupport operation of the facility 1202, according to someimplementations. The server 1304 may be physically present at thefacility 1202, may be accessible by the network 1302, or a combinationof both. The server 1304 does not require end-user knowledge of thephysical location and configuration of the system that delivers theservices. Common expressions associated with the server 1304 may include“on-demand computing”, “software as a service (SaaS)”, “platformcomputing”, “network-accessible platform”, “cloud services”, “datacenters”, and so forth. Services provided by the server 1304 may bedistributed across one or more physical or virtual devices.

One or more power supplies 1402 may be configured to provide electricalpower suitable for operating the components in the server 1304. The oneor more power supplies 1402 may comprise batteries, capacitors, fuelcells, photovoltaic cells, wireless power receivers, conductivecouplings suitable for attachment to an external power source such asprovided by an electric utility, and so forth. The server 1304 mayinclude one or more hardware processors 1404 (processors) configured toexecute one or more stored instructions. The processors 1404 maycomprise one or more cores. One or more clocks 1406 may provideinformation indicative of date, time, ticks, and so forth. For example,the processor 1404 may use data from the clock 1406 to associate aparticular interaction with a particular point in time.

The server 1304 may include one or more communication interfaces 1408such as input/output (I/O) interfaces 1410, network interfaces 1412, andso forth. The communication interfaces 1408 enable the server 1304, orcomponents thereof, to communicate with other devices or components. Thecommunication interfaces 1408 may include one or more I/O interfaces1410. The I/O interfaces 1410 may comprise Inter-Integrated Circuit(I2C), Serial Peripheral Interface bus (SPI), Universal Serial Bus (USB)as promulgated by the USB Implementers Forum, RS-232, and so forth.

The I/O interface(s) 1410 may couple to one or more I/O devices 1414.The I/O devices 1414 may include input devices such as one or more of asensor 136, keyboard, mouse, scanner, and so forth. The I/O devices 1414may also include output devices 1312 such as one or more of a displaydevice 1312(3), printer, audio speakers, and so forth. In someembodiments, the I/O devices 1414 may be physically incorporated withthe server 1304 or may be externally placed.

The network interfaces 1412 may be configured to provide communicationsbetween the server 1304 and other devices, such as the totes 1212,routers, access points 1310, and so forth. The network interfaces 1412may include devices configured to couple to personal area networks(PANs), local area networks (LANs), wireless local area networks(WLANS), wide area networks (WANs), and so forth. For example, thenetwork interfaces 1412 may include devices compatible with Ethernet,Wi-Fi®, Bluetooth®, ZigBee®, and so forth.

The server 1304 may also include one or more busses or other internalcommunications hardware or software that allow for the transfer of databetween the various modules and components of the server 1304.

As shown in FIG. 14, the server 1304 includes one or more memories 1416.The memory 1416 may comprise one or more non-transitorycomputer-readable storage media (CRSM). The CRSM may be any one or moreof an electronic storage medium, a magnetic storage medium, an opticalstorage medium, a quantum storage medium, a mechanical computer storagemedium, and so forth. The memory 1416 provides storage ofcomputer-readable instructions, data structures, program modules, andother data for the operation of the server 1304. A few examplefunctional modules are shown stored in the memory 1416, although thesame functionality may alternatively be implemented in hardware,firmware, or as a system on a chip (SoC).

The memory 1416 may include at least one operating system (OS) module1418. The OS module 1418 is configured to manage hardware resourcedevices such as the I/O interfaces 1410, the I/O devices 1414, thecommunication interfaces 1408, and provide various services toapplications or modules executing on the processors 1404. The OS module1418 may implement a variant of the FreeBSD™ operating system aspromulgated by the FreeBSD Project; other UNIX™ or UNIX-like variants; avariation of the Linux™ operating system as promulgated by LinusTorvalds; the Windows® operating system from Microsoft Corporation ofRedmond, Wash., USA; and so forth.

Also stored in the memory 1416 may be a data store 1420 and one or moreof the following modules. These modules may be executed as foregroundapplications, background tasks, daemons, and so forth. The data store1420 may use a flat file, database, linked list, tree, executable code,script, or other data structure to store information. In someimplementations, the data store 1420 or a portion of the data store 1420may be distributed across one or more other devices including theservers 1304, network attached storage devices, and so forth.

A communication module 1422 may be configured to establishcommunications with one or more of the totes 1212, sensors 136, displaydevices 1312(3), other servers 1304, or other devices. Thecommunications may be authenticated, encrypted, and so forth.

The memory 1416 may store an inventory management module 1424. Theinventory management module 1424 is configured to provide the inventoryfunctions as described herein with regard to the inventory managementsystem 120. For example, the inventory management module 1424 may trackitems 106 between different fixtures 102, to and from the totes 1212,and so forth. The inventory management module 1424 may access sensordata 1428. The sensor data 1428 may include weight data 122, non-weightdata 1430, and so forth. The non-weight data 1430 may include the doorsensor data 124, data obtained from other sensors such as cameras136(1), depth sensors 136(2), and so forth.

The data store 1420 may store other data such as the event data 1104,threshold data 1106, validity data 128, door status data 1112, validweight data 114, item data 130, physical layout data 132, interactiondata 134, and so forth.

The inventory management module 1424 may include one or more of a dataacquisition module 1426 and the analysis module 126. The dataacquisition module 1426 may be configured to acquire and accessinformation associated with operation of the facility 1202. For example,the data acquisition module 1426 may be configured to acquire sensordata 1428, such as the weight data 122, the non-weight data 1430 such asthe image data, and so forth.

The analysis module 126 may operate as described above, and isconfigured to determine the validity data 128 and generate interactiondata 134 using the valid weight data 1114. The analysis module 126 mayprocess other non-weight data 1430, such as the image data.

Processing of image data may be performed by a module implementing, atleast in part, one or more of the following tools or techniques. In oneimplementation, processing of the image data may be performed, at leastin part, using one or more tools available in the OpenCV library asdeveloped by Intel Corporation of Santa Clara, Calif., USA; WillowGarage of Menlo Park, Calif., USA; and Itseez of Nizhny Novgorod,Russia, with information available at www.opencv.org. In anotherimplementation, functions available in the OKAO machine vision libraryas promulgated by Omron Corporation of Kyoto, Japan, may be used toprocess the image data. In still another implementation, functions suchas those in the Machine Vision Toolbox for Matlab (MVTB) available usingMATLAB as developed by Math Works, Inc. of Natick, Mass., USA, may beutilized.

Techniques such as artificial neural networks (ANNs), active appearancemodels (AAMs), active shape models (ASMs), principal component analysis(PCA), cascade classifiers, and so forth, may also be used to processthe sensor data 1428 or other data. For example, the ANN may be atrained using a supervised learning algorithm such that objectidentifiers are associated with images of particular objects withintraining images provided to the ANN. Once trained, the ANN may beprovided with the sensor data 1428 to determination of similaritybetween two or more images, provide object identification, and so forth.

Other modules 1432 may also be present in the memory 1416 as well asother data 1434 in the data store 1420. For example, the other modules1432 may include an accounting module while the other data 1434 mayinclude billing data. The accounting module may be configured to assesscharges to accounts associated with particular users 112 or otherentities, while the billing data may include information such as paymentaccount numbers.

The processes discussed herein may be implemented in hardware, software,or a combination thereof. In the context of software, the describedoperations represent computer-executable instructions stored on one ormore non-transitory computer-readable storage media that, when executedby one or more processors, perform the recited operations. Generally,computer-executable instructions include routines, programs, objects,components, data structures, and the like that perform particularfunctions or implement particular abstract data types. Those havingordinary skill in the art will readily recognize that certain steps oroperations illustrated in the figures above may be eliminated, combined,or performed in an alternate order. Any steps or operations may beperformed serially or in parallel. Furthermore, the order in which theoperations are described is not intended to be construed as alimitation.

Embodiments may be provided as a software program or computer programproduct including a non-transitory computer-readable storage mediumhaving stored thereon instructions (in compressed or uncompressed form)that may be used to program a computer (or other electronic device) toperform processes or methods described herein. The computer-readablestorage medium may be one or more of an electronic storage medium, amagnetic storage medium, an optical storage medium, a quantum storagemedium, and so forth. For example, the computer-readable storage mediamay include, but is not limited to, hard drives, floppy diskettes,optical disks, read-only memories (ROMs), random access memories (RAMs),erasable programmable ROMs (EPROMs), electrically erasable programmableROMs (EEPROMs), flash memory, magnetic or optical cards, solid-statememory devices, or other types of physical media suitable for storingelectronic instructions. Further, embodiments may also be provided as acomputer program product including a transitory machine-readable signal(in compressed or uncompressed form). Examples of transitorymachine-readable signals, whether modulated using a carrier orunmodulated, include, but are not limited to, signals that a computersystem or machine hosting or running a computer program can beconfigured to access, including signals transferred by one or morenetworks. For example, the transitory machine-readable signal maycomprise transmission of software by the Internet.

Separate instances of these programs can be executed on or distributedacross any number of separate computer systems. Thus, although certainsteps have been described as being performed by certain devices,software programs, processes, or entities, this need not be the case,and a variety of alternative implementations will be understood by thosehaving ordinary skill in the art.

Additionally, those having ordinary skill in the art will readilyrecognize that the techniques described above can be utilized in avariety of devices, environments, and situations. Although the subjectmatter has been described in language specific to structural features ormethodological acts, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thespecific features or acts described. Rather, the specific features andacts are disclosed as illustrative forms of implementing the claims.

What is claimed is:
 1. A system comprising: a fixture comprising: aninsulated enclosure; a shelf within the insulated enclosure; a firstdoor configured to provide access to the shelf within the insulatedenclosure; a second door near the first door; a refrigeration unitconfigured to cool an interior of the insulated enclosure; aspring-loaded switch configured to generate first door sensor dataindicative of whether the first door is open or closed; a second sensorto generate second door sensor data indicative of whether the seconddoor is in an open state, a closed state, or transitioning between theopen state and the closed state; and a load cell configured to measureweight of a load on the shelf and generate weight data; and a computingdevice comprising: a memory storing first computer-executableinstructions; and a hardware processor to execute the firstcomputer-executable instructions to: access the first door sensor data;determine, using the first door sensor data, first door status dataindicative of the first door transitioning from a closed state at afirst time to an open state at a second time; access the second doorsensor data; determine, using the second door sensor data, second doorstatus data indicative of a configuration of the second door at one ormore of the first time or the second time; determine, using the firstdoor status data and the second door status data, that first weight datawhich corresponds to a time between the first time and the second timeis invalid; determine, using the first door status data and the seconddoor status data, that second weight data which corresponds to a timethat is after the second time is valid; generate interaction data usingthe second weight data; and update a quantity of a type of item at theshelf using the interaction data.
 2. A system comprising: a fixturecomprising: a shelf; a first door configured to provide access to theshelf; one or more first sensors to generate first sensor dataindicative of a state of the first door; a second door; one or moresecond sensors to generate second sensor data indicative of a state ofthe second door; and at least one load cell configured to measure weightof a load on the shelf and generate weight data; and a computing devicecomprising: a memory storing first computer-executable instructions; anda hardware processor to execute the first computer-executableinstructions to: process the first sensor data to determine first doorstatus data at a particular time; process the second sensor data todetermine second door status data at the particular time; and determine,using the first door status data and the second door status data,validity of the weight data obtained at the particular time.
 3. Thesystem of claim 2, the fixture further comprising: a communicationinterface; the memory storing second computer-executable instructions;and the hardware processor to execute the second computer-executableinstructions to: replace one or more bits of the weight data with thefirst sensor data to generate multiplexed data; and transmit, using thecommunication interface, the multiplexed data.
 4. The system of claim 2,wherein: the second door is near the first door.
 5. The system of claim2, the fixture further comprising: an enclosure with the shelf therein;and one or more heating devices to heat an interior of the enclosure. 6.The system of claim 2, the fixture further comprising: an enclosure withthe shelf therein; and a refrigeration mechanism configured to cool aninterior of the enclosure.
 7. The system of claim 2, the computingdevice further comprising a communication interface; the memory storingsecond computer-executable instructions; and the hardware processor toexecute the second computer-executable instructions to: determine theweight data obtained at the particular time is valid; and send, usingthe communication interface, the weight data obtained at the particulartime that is valid.
 8. The system of claim 2, further comprising thememory storing second computer-executable instructions; and the hardwareprocessor to execute the second computer-executable instructions to:designate the weight data obtained at the particular time as invalid. 9.The system of claim 2, further comprising: the memory storing secondcomputer-executable instructions; and the hardware processor to executethe second computer-executable instructions to: determine a valueindicative of how far open the first door is exceeds a threshold value;and designate the weight data obtained at the particular time as valid.10. The system of claim 2, wherein: the one or more first sensorscomprising one or more of a pressure sensor that detects pressure fromcontact of a hand, a proximity sensor that detects a presence of thehand, or a touch sensor that detects contact of the hand; and at leastone of the one or more first sensors is proximate to or at leastpartially within a handle of the first door and is configured togenerate the first sensor data that is indicative of use of the handle.11. The system of claim 2, wherein one of the one or more first sensorscomprises a camera having a field of view that includes at least aportion of the first door and the first sensor data comprises one ormore images acquired by the camera; and the memory storing secondcomputer-executable instructions; and the hardware processor to executethe second computer-executable instructions to: process the one or moreimages to determine a position of the first door.
 12. The system ofclaim 2, wherein the one or more first sensors comprise one or more of:a switch, a capacitive proximity sensor, an ultrasonic proximity sensor,an optical proximity sensor, a magnetic field sensor, an anemometer, atemperature sensor, a potentiometer, a force sensitive resistor, anoptical encoder, or a camera.
 13. The system of claim 2, wherein the oneor more first sensors are mounted to a front edge of one or more of theshelf or a structure supporting the shelf, further wherein the frontedge is proximate to the first door.
 14. A method comprising: generatingweight data using one or more load cells at a fixture that comprises ashelf and a first door; accessing first sensor data indicative of astate of the first door; accessing second sensor data indicative of astate of a second door; determining, using the first sensor data, firstdoor status data indicative of a configuration of the first door at aparticular time; determining, using the second sensor data, second doorstatus data indicative of a configuration of the second door at theparticular time; and determining, using the first door status data andthe second door status data, validity of the weight data obtained at theparticular time.
 15. The method of claim 14, further comprising:determining a first time at which the configuration of the first doorchanges between an open configuration and a closed configuration;determining a portion of the weight data that occurred between a secondtime and a third time, wherein the first time is between the second timeand the third time; and designating the portion of the weight data asbeing invalid.
 16. The method of claim 14, further comprising:determining a first time at which the configuration of the first doorchanges from an open configuration to a closed configuration;determining a portion of the weight data that occurred between a secondtime and the first time, wherein the second time is before the firsttime; and designating the portion of the weight data as being invalid.17. The method of claim 14, further comprising: determining the weightdata obtained at the particular time is invalid when the first door isclosed and the second door has changed between a closed configurationand an open configuration.
 18. The method of claim 14, furthercomprising: determining the weight data obtained at the particular timeis valid; and sending the weight data obtained at the particular timethat is valid to a computing device.
 19. The method of claim 14, furthercomprising: determining the weight data obtained at the particular timeis valid; and determining a weight value of a load at the fixture usingthe weight data obtained at the particular time.
 20. The method of claim14, further comprising: determining the weight data obtained at theparticular time is invalid; and determining a weight value of a load atthe fixture using the weight data obtained at the particular time, thedetermining the weight value of the load comprising one or more of:applying a filter to the weight data obtained at the particular time, orassigning a low confidence value to the weight value.